Blogs on WordPress.com have a new dashboard which highlights the community content more.
When working on some new AJAX features for bbPress and WordPress we’ve noticed that AJAX requests don’t seem to send HTTP_REFERER values. We check referrers as one level of protection against cross-site-scripting, or XSS, so when they’re not set we aren’t able to use that value. How are most people using AJAX protecting against XSS? It seems the same things we’re doing to make things easily accesible in a dynamic fashion are also opening new vectors for attack.
WordPress wins Forbes.com Best of Web and Favorite award for blog tools! “In February, open source blogging application WordPress came out with its release 1.5, and we’ve found that this release puts WordPress squarely ahead of its competition.” Our number of downloads has more than doubled since that was written. Hat tip: Niall Kennedy.
WordPress Database Backup plugin, can save to file or email it to you. Can combine with the WP-Cron plugin to run at specified intervals.
At the last IRC meetup the WordPress community asked for better search that included both the forums and the Codex and was integrated with the look and feel of the rest of the site. When I did this before it was horribly slow and it involved several queries across several different programs and MySQL hosts to get the results from the wiki, the forums, the blog, and then splice them together somehow. Later we switched to a plain Google site-search but they didn’t like the HTML we used for the search form so we took it down. Well after the meeting I remembered Yahoo Developer Network which had some sort of API for their search with a much higher limit than Google’s.
I went to the site to see how much of a pain it would be so I could start properly procrastinating, but I was taken aback by how incredibly easy it was to get an application ID and start getting the results back as simple XML. I began hacking on it right then. It was about 5 minutes to set up a search form with URIs the way I wanted, 7 minutes to get the XML and parse it out, 5 minutes to write in some paging, and then about 20 minutes tweaking the search page to make it look a little better. The result is the new search.wordpress.org WordPress Search.
It still needs some more work. There seems to be a dupe problem, which is actually a problem with our site, not Yahoo Search. I’d like to tweak the results to highlight newer topics more, or at leats allow for a date-based weighting. Finally I think it would be nice to include some WP-related blogs like Blogging Pro and Weblog Tools Collection in the results. Most importantly we now have a clean URI structure and home for searches which is abstracted from any piece of software or particular service provider. Yahoo deserves major kudos for opening up their information in such a free way and making it so easy that it’s taken me longer to write this post than start using their API.
“The Pulse of Freedom is a site published by the protesters at Martyrs’ Square, Beirut, from a tent city.” This is an inspiring story: “A group of Web masters, graphic design artists, writers, and photographers stayed up all night for several nights in a row putting the Web site together.” They chose WordPress. “As far as I’m aware this is the first Web site of its kind anywhere in the world. The leaders of a democratic revolution are openly blogging about their experience from the center of the action.” Echoditto writes about their part in Blogging from Beirut. “I am writing this post from a tent city in the Martyrs’ Square in central Beirut, a place which is filled with the energy and excitement of a burgeoning democratic movement.” Hat tip: Mike Carvalho via email.
Let me do my best to respond to the inquiries have been coming in, only some of these are direct quotes.
There is a shorter version of this available too.
Is this an April Fool’s joke?
Unfortunately not. If I was more clever perhaps I could make it a killer intro for one, but that’ll have to wait for next year.
What was your thinking behind accepting the advertising?
I called my sister last night to tell her about a present I found for her in the market and she interrupted me to say she saw my name show up in Google News a few times and started reading some of the articles. Before the phone card ran out she read me some headlines and my stomach sank. This is my first vacation and I almost didn’t even bring my laptop. (Luckily I talked myself into bringing it to do pictures.) I haven’t been on the internet since Monday and I obviously have a lot to catch up on. It was almost midnight when I found out and there was no access anywhere, so I woke up at 4:30 AM this morning to catch the first water bus to the airport and found some overpriced wifi, and here I am.
I have close to a thousand emails and countless blog posts and comments to go through, but I’ll try to synthesize everything and respond ASAP, I think it’s important because some people seem to be spinning things quite maliciously. If you have a specific question please send me an email and I’ll do my best to respond personally or on the blog, even if you’ve already decided I’m the scum of the earth.
As has been reported many places, and sent to me in a dozen emails, Dreamhost have integrated support for installing and upgrading WordPress through their administration interface. For many this is not new, some systems like Fantastico have had support for auto-installing WP for a while now, but the interface was cluttered and WordPress was just one of several dozen miscellaneous scripts. What’s really interesting about this decision is their thought process, quoted from their newsletter:
Why didn’t we make it so you could just install Movable Type itself with one click instead? Somewhat because Movable Type is a commercial product, and they would require us to do some sort of weird registration for all our users with them in order to even install the free version. Also, WordPress doesn’t require “rebuilding” your blog every time you publish. Mostly though, installing WordPress was eleven times easier for us.
Tyler Brekko sent me screenshots from the entire installation process, including emails from the “Happy DreamHost WordPress Robot” and it’s kinda neat. They take advantage of a few structural decisions in WordPress and on wordpress.org to make the process very smooth. You can read a bit in their knowledge base. What surprised me the most was the sheer number of people who emailed me about this, DreamHost must have a ton of users. Here are some other posts around the web about this:
Thanks to everyone who wrote in about this!
(Also known as Announcements Two and Three. See part one.)
Matt Mullenweg may be underage, but he knows how to get into a bar.
That’s the opening line from the Houston Press feature called The Blog Age, subtitled “Matt Mullenweg helps usher in the real information revolution, one Web log at a time.” Modest, eh? It should be on newstands tomorrow so if you’re in Houston pick up a copy. Otherwise read it online.
Though it’s a little embarassing for me in parts, it’s a really good article that covers everything from Open Source to my fellow H-Town bloggers to political blogging at the national conventions. I’ve been talking to Cathy (the author) off and on since August and the strangest thing is how much has changed since we first met in a small coffee shop in Montrose. There are a few points I’d like to expand on or clarify but I’m exhausted from today’s travel right now.
The Other Big News
If you’ve read the article already you’ll notice that it breaks a juicy tidbit that hasn’t been published before: I’ve accepted a job with CNET Networks. That’s the reason I’m moving to San Francisco. There were a lot of ways things could have gone and honestly I wasn’t even planning on leaving Houston originally, but over the course of the past 3-4 months I’ve been talking to CNET and an opportunity came up I just couldn’t pass. As it says in the article part of what they’re paying me for is working on WordPress just like I do now. The rest of my time I’ll be working on various projects, most of them probably so top-secret I can’t breathe a word in advance.
The reason I’m excited about working with CNET is how what I’ll be doing meshes with my thoughts and ideals regarding Open Source, standards, and communities. My principles aren’t changing just because my paycheck is. You can expect the same sort of content here on PhotoMatt.net you’ve seen in the past — namely unfiltered personal thoughts, jabs, and observations not connected with any corporate entity. This is obviously a pretty significant move for both myself and WordPress so I’ll do my best to entertain any questions you may have in the comments.
A few comments about some of the code changes in 1.3. Very nice to see people appreciating some of the hard work we’ve put into this iteration. In a perfect world we could stop the clock and rewrite large portions of the code from scratch, but that would take a long time and break a lot of things in the process. All programmers want to do this, it’s our weakness, but every time I get this urge I think of Netscape and how devastating their rewrite downtime was. We’re making some substantial changes but doing it gradually while introducing new features and responding to users needs.
For example, in 1.3 “the loop” is called very differently but it’s completely backward compatible with everyone’s 1.2 loop code. We deprecate things over time so any structural changes that need to be made come gradually for people upgrading, there’s nothing to drastic every time. This also saves a huge amount of time in support. (Regular users don’t want to have to redo their templates, hackers don’t want to relearn code they already knew.) Same for the new theme system we’re introducing, it adds a lot of flexibility, radically changing how the front end of WordPress operates (like plugins for templates and styles) but all the new stuff is completely optional. I’ve transitioned most of my custom code into a personal “theme” that makse upgrading a lot easier for me (which is good because I do it almost daily).
Next time you get the urge to rewrite from scratch think about the testing your code has gone through, all the edge cases that have already been addressed, the existing installed base, and how many new bugs you’ll introduce with the from-scratch code.
WordPress 1.2 was downloaded over a hundred thousand times. About two-thirds of that was through the new download system where we can track stats better. It’ll be interesting to see the download rate of 1.2.1 (and subsequently 1.3).
It’s great to see one labor of love inspiring another. I can’t wait to see what these talented people do when the theme system for 1.3 is released.
Online Journalism Review writes Bloggers Declare War on Comment Spam, but Can They Win? I’m not sure what that has to do with journalism, but they talk to the same old people and read the same old sites and (not surprisingly) come to the same old tired conclusions. I’m trying to figure it out because I like everyone the article refers to and the article itself is well-written, but it feels very contrived. I think it may be because it draws a lot from blog material a year or more old, and selectively, like the writer had an agenda and Googled until there were enough quotes to fill the space. For example Mark Pilgrim’s blog is called “comment-free” when the entry on the front page for the last three weeks clearly has comments. Is it too much to ask to look at the front page of a blog you’re quoting? The article talks about Blogger redirecting URIs but not about Blogger’s registration aspect. It talks about Typekey but not the PATRIOT act. (Totally kidding there.)
You probably saw this coming from me, but most of all I think it’s silly that they don’t mention a single one of the dozens of other blogging systems that deal effectively with these issues every day. You can’t discuss the Movable Type spam epidemic without talking about people like Molly who tried everything out there including MT-Blacklist to no avail, then switched software and got on with their lives. There is a lot more to the story, but that’s been the conversation over the past year and a lot has come of it. The essence of blogging is communication and comments are here to stay, it’s just a matter of moderation.
Recently it leaked on a blog (there are few secrets in Open Source) that elements from a design known as “Kubrick” by Michael Heilemann would be incorporated into the default template for the next version of WordPress. Kubrick is many things: a design, a set of templates, some plugins, and a removal of a lot of cruft currently in the default template. It makes things much friendlier for readers. Best of all Michael released everything under the GPL and submitted it to WordPress for inclusion. After it had had several iterations I checked it out and saw a lot of great ideas that would make WordPress a better product, especially for new users. Even though no decisions had been made and no code had been committed, a number of questions were raised in people’s minds. A thread was started in the forums that I’m not even going to link to because it’s not worth reading past the first page, if that. Many people seemed to misunderstand what was going to be incorporated and what wasn’t, even though that was stated pretty clearly in the original blog post.
Michael is primarily a designer, not a coder, and coding things in a way that works on the variety of platforms and setups that WordPress itself does is hard, so there are issues with that in the templates Michael has released. WordPress devs have a lot of experience with those issues, however, and anything added to the core will work just as well (if not better) than WordPress does now. Several others questioned the inclusion of graphics in a template. If graphics were included, how would people be able to edit it? We can’t expect people to have graphics editors, so if graphics are included in the final template (that hasn’t been determined yet) I’ve committed to providing an online interface on wordpress.org for people to customize the graphics to match their color choices without needing any software beyond a web browser. There were some questions about the CSS being used in Kubrick, but the CSS used for it in WordPress won’t be the same and will be treated like any change to the WordPress code, that is it will go through the normal QA process and be tested across platforms by the developers and the few dozen or so people who keep up with the nightly builds, and then extensively tested by the hundreds that use the beta releases once we enter that phase for 1.3. Any problems will be treated as bugs and fixed as such. Watching trends on the forums and continuing a high level of support is very important to everyone.
The problem was after all this was explained the thread continued long after all these questions had been answered with everyone talking past each other. If it shows anything it’s that people can be very passionate about the smallest of things. It’s interesting to note that while this all was occuring what has actually happened in WordPress development in the last week: Dougal wrote a plugin to slow down spambots, literally; Alex made a new style for the styles page; Kitten sent in another comment moderation plugin that’s going to be included in the core; Craig Hartel and Kevin Francis (amoung many others) did some great work on the new wiki; Michel is refactoring the XML-RPC code; we started the process of moving to a better source control system; Ryan is coding too much cool stuff to mention, but the next version of WP be the easiest to customize and template ever. That’s just off the top of my head, there’s lots of other exciting developments happening.
In other words, life moved on. It showed up on a few blogs, but that’s a price of popularity: bad news gets more buzz than good. Numerous examples are in the checkout line of every supermarket. (Not to mention the blogosphere.)
So what’s the state of the WordPress community today? I’d say it’s better. The number of people who actually got out-of-hand was only a handful, and personally I’m ready to apologize and move on. I’ve never been good at holding grudges. The things that make the WordPress community great haven’t changed, and several lessons have been learned. Hundreds of new WordPress blogs have been started, testimonials and donations keep coming in, I’ve noticed more people helping out on the forums, and best of all there’s a healthy amount of disagreement keeping the project young.
Through Carthik’s post A minor debate I came across this thread talking about comment feeds. The thread is a little funky and Carthik is obviously enthusiastic, but what stood out is Anil‘s comment “If there’s enough demand from users for it, we’ll include them in MT as part of the package.” Which prompts the question, how much demand from users for this was there in WordPress? Do we just bloat the willy-nilly with every idea that comes down the line? The answer is in two parts:
First, a great deal of thought and deliberation goes into every feature we include with WordPress, particularly the ones enabled by default. One guiding force of WordPress is that every release is faster than the one before that, and to do that you have to optimize ruthlessly and be very wary of any bloat in the code. So far we’ve been very successful with this: WordPress is at least 3 times faster than b2 was and we still have added features that other systems are just beginning to catch up with. With comment feeds there is the immediate benefit of people being able to subscribe to any thread on any WordPress site in the world, but there is a further benefit of bootstrapping a technology of which the benefits are just beginning to be fully realized. Feedster can index not just every post on a WordPress blog, but every comment as well. Aggregator developers may not have gone to the trouble of supporting
<wfw:commentRSS> for just a few custom feeds, but now I can point fifteen thousand blogs using it to point to a countless number of comment feeds.
The market might not be demanding a feature yet, but if you just wait for the market to decide it wants something you’ll always be following and never leading.
Second, a great idea can come from a single user. Pure numbers are a factor when considering new feature suggestions, but most good ideas stand on their own merits. Innovation usually comes from the places you least expect it. If I remember correctly we had about a dozen or so people interested in comment feeds that I knew of, but it really could have been one. It was an idea that made a lot of sense within the stated goals of WordPress and didn’t cost anything to add. Alex got the code together and it was in the next release. It’s been improved a couple of times, and now you can add
/feed/ to any permalink (or category page, etc) in WordPress and get the feed you want. Users that didn’t know they wanted comment feeds before are thanking us now. Eventually all modern blog software will support comment feeds, and WordPress will have moved on to something else new.
Under the Iron has an old interview with Scott Johnson that is a good read. Now scroll down to the comments. Dozens and dozens of spam comments. I see this over and over again on MT and s9y sites. What’s terrible is these pages are just as dangerous as dedicated spam blogs. Think about it: I shouldn’t even be linking to it now.
Alex told me the other day about a new type of comment spam he’s been seeing: comments that link to normal blog entries. Well known blogs like Mozillazine. As advanced as tools like MT Blacklist have become, they’re pretty useless in cases like this. Are you going to blacklist Dave Sifry? Molly.com used to have spam comments on her site all the time. Even though she spent a lot of time and effort dealing with them (a daily chore) they only need to be there long enough for Googlebot to index them for the harm to be done. I’m not dogging on MT here, it’s just that there are tens of thousands of MT blogs out there who don’t have any protection and the spammers are targetting them mercilessly. Domain blacklists don’t scale (spammers can have thousands of domains easily and hijack innocent domains) and centralized registration hasn’t shown to be effective except against people who don’t like centralized registration, a group that doesn’t include spammers.
People used to say that WordPress doesn’t get spam comments because it’s not popular enough. I don’t think this argument holds water anymore. It’s true that MT has three to four times as many blogs as WordPress, but Serendipity has an order of magnitude fewer blogs than WP and is highly targetted by spammers. I think WordPress has, through design and luck, done a lot of things right with regards to comment management in general. First we respond to the problem in the core code quickly. Moderation and blacklisting has been in the core for half a year now. All of the WordPress developers are bloggers as well so we’re pretty sensitive to new techniques in use by the spammers. When early versions of WordPress 1.0 advertised moderation was on spammers instantly adapted to that and started searching for blogs that didn’t have the phrases we used, so in the next nightly build for testers I had changed how that worked so it couldn’t be targeted anymore. Then in 1.2 we expanded the already successful moderation to allow powerful regular expressions and target not just the content but things like number of links in a post. Let’s say that somehow two hundred spam comments did get on your blog, which would never happen in the first place because we’ve had throttling for over a year now, you can easily delete hundreds of spam comments at once in under five clicks. We’re not sitting still either, version 1.3 will have emergent registration based on code originally written by Kitten so there is a type of automatic whitelisting going on that spammers can’t duplicate because it uses email addresses like a secret key and WordPress never reveals your email address. (So Dave and Mark, stop leaving fake ones!) The code will be flexible enough to adapt for GPG signing for the ultra-geeky in the audience.
Any of these things by themself wouldn’t be very effective, and each method I’ve listed has its flaws and weaknesses and I know them. Which brings us to what I think the real reason WordPress, despite its explosion of popularity, still doesn’t get the level of spam other tools do: it’s more trouble than it’s worth. WordPress, to spammers, is an unpredictable and moving target. We’re not resting on our laurels, we have another exciting feature-filled release coming just a few months after the landmark version 1.2. The WordPress moderation system can be be toggled to manual mode, which is 100% effective at catching spam, or triggered only when something is suspicious. We’re committed to keeping the cost high and the reward uncertain for spammers which means you don’t have to wake up every morning to filth on your weblog as well as in your inbox. You can focus on what draws us all to this medium, writing and genuine interaction. Here’s a quote from Molly from a comment she left on Keith’s site:
I wanted open comments. In my situation, MT, despite the wonderful Jay Allen personallyhelping me on an almost daily basis to deal with comment spam, I was a major target. My ISP refused to continue dealing with me because the server molly.com resided on was brought to its knees twice due to spam floods. I was spending up to two hours PER DAY to undo the spam much less post.
Since switching to WP, I’ve had exactly five emails sent to me automagically for moderation. 3 of them were spam, 2 were just enthusiastic posts with multiple links from a reader.
Either way, I had instantaneous access to accept or delete those posts.
That’s the sort of thing that is incredibly rewarding about working on WordPress. Knowing that your work makes it easy for someone else to do what they love is one of the greatest feelings in the world. No amount of money or recognition can ever match that.