Through Carthik’s post A minor debate I came across this thread talking about comment feeds. The thread is a little funky and Carthik is obviously enthusiastic, but what stood out is Anil‘s comment “If there’s enough demand from users for it, we’ll include them in MT as part of the package.” Which prompts the question, how much demand from users for this was there in WordPress? Do we just bloat the willy-nilly with every idea that comes down the line? The answer is in two parts:

First, a great deal of thought and deliberation goes into every feature we include with WordPress, particularly the ones enabled by default. One guiding force of WordPress is that every release is faster than the one before that, and to do that you have to optimize ruthlessly and be very wary of any bloat in the code. So far we’ve been very successful with this: WordPress is at least 3 times faster than b2 was and we still have added features that other systems are just beginning to catch up with. With comment feeds there is the immediate benefit of people being able to subscribe to any thread on any WordPress site in the world, but there is a further benefit of bootstrapping a technology of which the benefits are just beginning to be fully realized. Feedster can index not just every post on a WordPress blog, but every comment as well. Aggregator developers may not have gone to the trouble of supporting <wfw:commentRSS> for just a few custom feeds, but now I can point fifteen thousand blogs using it to point to a countless number of comment feeds.

The market might not be demanding a feature yet, but if you just wait for the market to decide it wants something you’ll always be following and never leading.

Second, a great idea can come from a single user. Pure numbers are a factor when considering new feature suggestions, but most good ideas stand on their own merits. Innovation usually comes from the places you least expect it. If I remember correctly we had about a dozen or so people interested in comment feeds that I knew of, but it really could have been one. It was an idea that made a lot of sense within the stated goals of WordPress and didn’t cost anything to add. Alex got the code together and it was in the next release. It’s been improved a couple of times, and now you can add /feed/ to any permalink (or category page, etc) in WordPress and get the feed you want. Users that didn’t know they wanted comment feeds before are thanking us now. Eventually all modern blog software will support comment feeds, and WordPress will have moved on to something else new.

Under the Iron has an old interview with Scott Johnson that is a good read. Now scroll down to the comments. Dozens and dozens of spam comments. I see this over and over again on MT and s9y sites. What’s terrible is these pages are just as dangerous as dedicated spam blogs. Think about it: I shouldn’t even be linking to it now.

Alex told me the other day about a new type of comment spam he’s been seeing: comments that link to normal blog entries. Well known blogs like Mozillazine. As advanced as tools like MT Blacklist have become, they’re pretty useless in cases like this. Are you going to blacklist Dave Sifry? Molly.com used to have spam comments on her site all the time. Even though she spent a lot of time and effort dealing with them (a daily chore) they only need to be there long enough for Googlebot to index them for the harm to be done. I’m not dogging on MT here, it’s just that there are tens of thousands of MT blogs out there who don’t have any protection and the spammers are targetting them mercilessly. Domain blacklists don’t scale (spammers can have thousands of domains easily and hijack innocent domains) and centralized registration hasn’t shown to be effective except against people who don’t like centralized registration, a group that doesn’t include spammers.

People used to say that WordPress doesn’t get spam comments because it’s not popular enough. I don’t think this argument holds water anymore. It’s true that MT has three to four times as many blogs as WordPress, but Serendipity has an order of magnitude fewer blogs than WP and is highly targetted by spammers. I think WordPress has, through design and luck, done a lot of things right with regards to comment management in general. First we respond to the problem in the core code quickly. Moderation and blacklisting has been in the core for half a year now. All of the WordPress developers are bloggers as well so we’re pretty sensitive to new techniques in use by the spammers. When early versions of WordPress 1.0 advertised moderation was on spammers instantly adapted to that and started searching for blogs that didn’t have the phrases we used, so in the next nightly build for testers I had changed how that worked so it couldn’t be targeted anymore. Then in 1.2 we expanded the already successful moderation to allow powerful regular expressions and target not just the content but things like number of links in a post. Let’s say that somehow two hundred spam comments did get on your blog, which would never happen in the first place because we’ve had throttling for over a year now, you can easily delete hundreds of spam comments at once in under five clicks. We’re not sitting still either, version 1.3 will have emergent registration based on code originally written by Kitten so there is a type of automatic whitelisting going on that spammers can’t duplicate because it uses email addresses like a secret key and WordPress never reveals your email address. (So Dave and Mark, stop leaving fake ones!) The code will be flexible enough to adapt for GPG signing for the ultra-geeky in the audience.

Any of these things by themself wouldn’t be very effective, and each method I’ve listed has its flaws and weaknesses and I know them. Which brings us to what I think the real reason WordPress, despite its explosion of popularity, still doesn’t get the level of spam other tools do: it’s more trouble than it’s worth. WordPress, to spammers, is an unpredictable and moving target. We’re not resting on our laurels, we have another exciting feature-filled release coming just a few months after the landmark version 1.2. The WordPress moderation system can be be toggled to manual mode, which is 100% effective at catching spam, or triggered only when something is suspicious. We’re committed to keeping the cost high and the reward uncertain for spammers which means you don’t have to wake up every morning to filth on your weblog as well as in your inbox. You can focus on what draws us all to this medium, writing and genuine interaction. Here’s a quote from Molly from a comment she left on Keith’s site:

I wanted open comments. In my situation, MT, despite the wonderful Jay Allen personallyhelping me on an almost daily basis to deal with comment spam, I was a major target. My ISP refused to continue dealing with me because the server molly.com resided on was brought to its knees twice due to spam floods. I was spending up to two hours PER DAY to undo the spam much less post.

Since switching to WP, I’ve had exactly five emails sent to me automagically for moderation. 3 of them were spam, 2 were just enthusiastic posts with multiple links from a reader.

Either way, I had instantaneous access to accept or delete those posts.

That’s the sort of thing that is incredibly rewarding about working on WordPress. Knowing that your work makes it easy for someone else to do what they love is one of the greatest feelings in the world. No amount of money or recognition can ever match that.