Tomorrow, along with the rest of Automattic, I am heading down to La Paz, Mexico for the twice yearly gathering of the company. The virtual company thing is great, but I still think it’s important to get everyone together a few times a year to connect and plan out the next 6-12 months.
Category Archives: Automattic
Sun Isn’t Relevant to Startups
Preface: I don’t write critically about a company unless I think they have some glimmer of getting better, however small that may be.
I just watched another Scoble Show interview with Jonathan Schwartz, and it reminded me how frustrated I continue to be with Sun, particularly their Startup Essentials program. He seems to be complaining about the perception of startups that the Sun platform is completely wrong for a new generation of startups, a perception I would strongly agree with.
To illustrate, here is a timeline of my experience with the Startup Essentials program.
November 3rd, 2006: Drinking the kool-aid at Startup Camp, register on behalf of Automattic for the Startup Essentials program.
November 8th, 2006: I receive an email addressed to “Mr Muellenweg” (wtf?) saying “Due to the volume of interest in this exciting new program, it is taking us longer than expected to review your application. We will let you know your status within the next few days — thanks for your patience.”
Late-November: I talk for about an hour to a nice guy named Eric Cosway doing research about what I thought of Startup Camp and Sun. I tell him about the trouble I’ve had registering for Startup Essentials, he says he’ll note that and pass it on.
3 months since: Complete silence, punctuated by frequent Sun press releases and AP stories about how they’re great for startups, probably millions of dollars of PR.
Seeing their constant press releases and hot-air filled announcements around “Web 2.0” just makes Sun seem that much more out of touch with reality and desparate to jump on hype which has already passed.
I’m not writing this, as Joyent did, with the hope of connecting with someone at Sun that will take my money. I’ve given up, I’ve lost hope, and our business has moved on. We’re growing fast and adding infrastructure faster, all on a tight budget, and I’m less inclined to depart from the LAMP-based architecture that Digg, Flickr, Wikipedia, and dozens of other sites I respect use.
This is also what I’ve started advising other startups, if we had trouble getting anything going I can’t imagine what trouble a brand new startup with a smaller profile would go through. (An exception to this is going through Joyent, which has packaged Sun’s stuff so well Sun should just buy them.)
Personally I’m far more excited about what Amazon is doing these days.
Update: In a move most would never expect of a public company CEO, Jonathan Schwartz has responded. More than that, he has apologized in a human and personal way that is utterly admirable. I’m going to work on a followup to this entry.
Update 2: I’ve posted the followup.
Economical Grids?
My friend Jason is blogging about Joyents grids and linked to one of their new whitepapers. The cost structures in the whitepaper were interesting to me, to give the executive summary for $2425 a month they got in aggregate 12 GB of RAM, 130GB of storage, and 750GB of public transfer. Some of the storage was block, and some was “distributed NFS mounted.” In systems terms, this came down to 800/mo for Big-IP load balancing, 1000/mo for 8 web servers, and 500/mo for 2 DBs.
Since a lot of startups ask me for hardware advice, I was curious this setup would compare to one of the better dedicated providers, LayeredTech. As disclosure, Automattic currently has 8 servers with Joyent that we got when WP.com first started (and they were called Textdrive) and we have 50+ with Layered Tech. Both are active, we serve blog traffic from Dallas (LT) and the main site/tags traffic from San Diego (Joyent).
However this raises an interesting question: If we had $2425/mo to burn and were just starting out, which would be a better choice?
We have the Joyent costs laid out there in a typographically-correct whitepaper, so I decided to dig into LayeredTech’s slightly-1999 website to see what would be comprable. First I started with the web nodes, their best deal currently seem to be the AMD 275 dual core with 2GB RAM, 250gb HD for 175/mo. Let’s get 10 of those. Let’s get two more but with dual 73gb SCSI mirrored drives for the DBs, at 190/mo. We need a file server, something CPU light, I went for this one and put two mirrored 500gb drives in it for 159/mo. Finally we need some load balancers. Two million hits a day is only 23 requests a second, which my laptop couuld probably do, but let’s get two 3800s at 109/mo each and we can put Pound + Wackamole on them for high availability. Those can easily balance probably about 30-40mbps of traffic each.
DoubleMattic
Now with Twice the βMatticβ – Matt Thomas joins Automattic full-time. We’re still figuring out what to call him, since “Matt” is obviously taken. π
PC World Blog Tips
PC World – New Tools Help Take Your Blog to the Next Level. “Top-flight bloggers are increasingly moving to WordPress, Automattic Productions’ no-cost, open-source blogging software. Unlike the basic blogging tools found in Blogger, Microsoft’s Windows Live Spaces, and Yahoo 360, WordPress offers tons of plug-ins and widgets for customizing your blog.”
BloggingPro Interview
Over the weekend I did an email interview with BloggingPro. Dropped a few hints about the future, so if you’re curious what’s in store for WordPress and Automattic check it out.
Toni Interview Video
WordPress.com VIP
Lloydomattic
Feeling Automattic. Lloyd Budd, of Flock/Mozilla fame, has joined Automattic full-time. π
New Servers
We’re looking to expand into a third datacenter for WordPress.com and related Automattic stuff. I’ve been asking around for quotes, but it occurred to me I should blog what we’re looking for because someone reading might be able to provide what we need, or know someone who could. One line summary of what we’re looking for: 22 boxes, AMD 3800 or similar, cheapest HD, 1-2gb of RAM, Debian AMD64 (this is important), private and public 100mbps network. I’d be happy to post more info in the comments if anyone wants it. A per-server monthly price including space and bandwidth is what we’re looking for.
Automattic Adams
Web Scaling Books
I’ve read two really good books on scaling large-scale web applications lately: Building Scalable Websites by Cal Henderson and Scalable Internet Architectures by Theo Schlossnagle. (Original titles, eh?) Both dispel common myths and misconceptions about scaling. While neither maps directly to the approaches we’re taking at Automattic, they’re both must-read for developers approaching or past the million/day pageview mark. I only wish I had more time to discuss my thoughts on the various concepts in the books—particularly Cal’s.
Bryan Veloso
Podsession Responses
There have been two interesting responses to the podcast I did with Om and Niall the other week. The first was Scott Johnson who responded in a podcast. As I expected, most people are taken aback by my statement to “let the engineers pick” what language and enviroment you use for your product. I think there is one important assumption that wasn’t articulated in that statement: you have brilliant engineers and you trust them. As a psuedo-engineer, I find it insulting when people suggest engineers are unable to factor anything other than their selfish language preferences, things like loaded costs, hardware costs, platforms, long-term viability, hiring, etc are simple variables that can be considered by any intelligent person. If anybody in Automattic came to me that was writing a tool in Python, C, Perl (it’s happened) or whatever, I might ask a question or two but at the end of the day I know they’re able to weigh the costs and benefits just like I would. If you’ve hired an engineer that isn’t able to make these decisions as well or better than you, then you’ve already lost the battle and over time more and more of your time will be spent plugging holes in a descent to mediocrity.
The second response was on the Pronet blog which in an amazing feat of blogging acrobatics managed to mention and link every single person tangentially associated with the podcast except me, even though I’m quoted in every heading. The Google Pages example is brought up again to illustrate that all the hardware in the world sometimes solve a scalability problem, but I still think that’s faulty because none of us had any idea why Pages was slow when it launched, it could have been a faulty router for all we know. Pronet responds to “Go with what your happiest working with” with a set of points to consider for a language, but again with the right people none of that matters. Happiness, in all things not just the language, should be the number goal and metric for everything in an early-stage startup. Happy engineers work smarter, longer, more efficient, attract better candidates, and have a better quality of life. (A corollary is that if you’re already set on a language path, don’t hire anyone who isn’t thrilled with working in that language.) For an example of how this can work in a really extreme case, I suggest everyone read the story of Viaweb and Lisp. (Another talk.)In my mind Lisp is a ridiculous language to build a web application in, but to them and their engineers it was heaven and they had better products earlier than their competition as a result of their unusual choice.
(As an aside, I wonder how many people said the same thing about Ruby for web apps before David Heinemeier Hansson, Rails, and 37signals, or even about PHP before Yahoo and Wikipedia? An example (and a little bit of promotion) is better than a thousand whitepapers.)
Open Source Legal Docs
Not technically open source, because I don't know which license is best for regular text, but I just put a Creative Commons Sharealike license on the WordPress.com terms of service and Automattic privacy policy. People were stealing them anyway, might as well make it legit. π Feel free to grab bits and pieces and search/replace your company/project in. If you want to throw us a link as a thank you, I'd be flattered.
Awesome Spellchecker
We now have inline-spellchecking on WordPress.com, and it will be in core for WordPress 2.1. It works a lot like Gmail, certainly one of the best I’ve seen in any web editor. Automattic sponsored the good folks at Moxiecode to develop this feature. All GPL, natch.
Hours and Work
There’s no correlation between hours worked and work done. I think this is why traditional corporate structures are starting to crack at the seams, and the distributed model companies like Automattic, MySQL, SocialText, and many others use will start to gain real legs and acceptance. The best example of this was at a place I used to work: after lunch everything seemed to shut down. Several people obviously got very sleepy after lunch and would spend 2-4 hours of the afternoon on auto-pilot. (This was me sometimes too.) It would have been infinitely better for them to take a one hour nap and get back to productive work than spend 3 hours in an obviously hampered state. Happy, healthy, well-rested people work orders of magnitude better.