Lydia LeongA recent interview of some Twitter developers, on Twitter’s use of Scala has touched off a fair amount of controversy in the Ruby community, and prompting Todd Hoff of the High Scalability to muse on an interesting statement: At some point, the cost of servers outweighs the cost of programmers.
We all know that the scripting languages that are frequently favored in Web development today — Ruby, Python, and PHP — do not perform as well as Java, and Java in turn can be outperformed by well-written native C/C++ code. However, these popular dynamic programming languages typically lead to better programmer productivity. The argument has been that it’s more cost-effective to have more productive developers, than it is to buy less infrastructure. There is a point, though, when that scale equation can be flipped on its head — when the cost of the servers, due to the performance sacrifices, gets too high. (I would add that you can’t look at simple hardware spend alone, either. You’ve got a infrastructure TCO to look at. It’s not just about more people to maintain more servers, either — that equation is not linear, as a sysadmin can manage more systems if they’re all identical and there are good automation tools. But systems that are struggling due to performance issues soak up operations time with daily firefighting.)
Twitter’s developers are not advocating that people abandon what they know and love, but they’re forging a new path for themselves, with an open-source language developed in academia. Scala can be compiled to either Java or .NET bytecode, allowing it to interoperate bidirectionally with Java and CLR code; this is important for driving adoption because programmers generally like to work with languages that have a solid base of libraries (i.e., someone else has conveniently done the work of producing code for commonly-needed capabilities), and because this makes it possible for Scala to leverage the existing tools community for Java and .NET. Scala’s equivalent of Rails, i.e., a convenient framework, is Lift.
Scala doesn’t have much adoption now, but it’s worth noting that the rapid pace of Web 2.0 innovation is capable of driving extremely fast uptake of things that turn out to solve real-world problems. (For comparison: Not long ago, practically no one had heard of Hadoop, either, but it’s built quite a bit of buzz now.) That’s important for anyone contemplating the long-term future of particular platforms, particularly APaaS offerings that are tied to specific programming languages. The favored platforms can and do change in a tidal fashion — just look at the Google trend graph for Ruby on Rails to see just how aggressively interest can increase over a single year (2005 to 2006).
As a coda to all of this, Twitter’s Alex Payne has a smart blog post, noting that social media fills the vacuum between peer-reviewed journals and water-cooler conversations, yet deploring the fact that in these mediums, emotion can rule over what is measurable. The takeaway — whether you’re an IT manager, a marketing manager at a vendor, or an investor — from my perspective, is this: There’s an emotional context to programming language choice. These are not merely technical communities; these are fandoms, and they form part of a developer’s self-identity.
Category: Infrastructure Tags: appdev, people, scalability
2 responses so far ↓
1 Justin D-Z April 6, 2009 at 12:12 pm
I’m a big Hacker News fan and a Ruby developer on the side, so I’m well aware of the emotional aspects of programming. Generally, Ruby ends up in trouble in longer-term “Enterprise” environments. Now, this isn’t necessarily a failing of Ruby, but in the case of Twitter they refactored the back-end to C, I believe, as part of their last wave of efforts to really make the platform operational according to growth. Martin Fowler would be proud.
I often wonder if this model where a dynamic, terse, highly productive language is used on in the view layer and the back-end is based on something slower, heavier and with a larger eco-system of performance tools (e.g. Java) is a good model and whether the Ruby argument is really to just refactor your way there because starting out with that model is premature optimization and you shouldn’t assume you’ll need to go there until you do. In which case, Twitter just didn’t or couldn’t see it coming early enough.
Scala is an interesting thing to watch. But I would suggest that functional programming languages much like most dynamic languages are a hard sell to large IT shops. Twitter doesn’t have to deal with those issues yet and may never.
2 Nick Jones April 7, 2009 at 7:11 am
But one thing IT shops will have to worry about a lot more is green IT. Inefficient programs eventually equate to more electricity consumption, which has both opex implications and may also conflict with wider corporate environmental initiatives.