Lydia LeongI’ve been thinking about the way that the economics of cloud computing infrastructure will impact the way people write applications.
Most of the cloud infrastructure providers out there offer virtual servers as a slice of some larger, physical server; Amazon EC2, GoGrid, Joyent, Terremark Enterprise Cloud, etc. all follow this model. This is in contrast to the abstracted cloud platform provided by Google App Engine or Mosso, which provide arbitrary, unsliced amounts of compute.
The virtual server providers typically provide thin slices — often single cores with 1 to 2 GB of RAM. EC2′s largest available slices are 4 virtual cores plus 15 GB, or 8 virtual cores plus 7 GB, for about $720/month. Joyent’s largest slice is 8 cores with 32 GB, for about $3300/month (including some data transfer). But on the scale of today’s servers, these aren’t very thick slices of compute, and the prices don’t scale linearly — thin slices are much cheaper than thick slices for the same total aggregate amount of compute.
The abstracted platforms are oriented around thin-slice compute, as well, at least from the perspective of desired application behavior. You can see this in the limitations imposed by Google App Engine; they don’t want you to work with large blobs of data nor do they want you consuming significant chunks of compute.
Now, in that context, contemplate this Intel article: “Kx – Software Which Uses Every Available Core“. In brief, Kx is a real-time database company; they process extremely large datasets, in-memory, parallelized across multiple cores. Their primary customers are financial services companies, who use it to do quantitative analysis on market data. It’s the kind of software whose efficiency increases with the thickness of the available slice of compute.
In the article, Intel laments the lack of software that truly takes advantage of multi-core architectures. But cloud economics are going to push people away from thick-sliced compute — away from apps that are most efficient when given more cores and more RAM. Cloud economics push people towards thin slices, and therefore applications whose performance does not suffer notably as the app gets shuffled from core to core (which hurts cache performance), or when limited to a low number of cores. So chances are that Intel is not going to get its wish.
Category: Infrastructure Tags: appdev, Cloud
2 responses so far ↓
1 Dan Sholler October 17, 2008 at 1:25 pm
Lydia,
Isn’t the bias toward smaller things inherent in the model? The economies of scale are only realized if there are many small things that have different load requirements. If there are large things (or many small things with precisely the same load requirements, which is essentially the same) , then they have to be offset by a bunch of smaller things with precisely opposite load requirements, or I am in trouble. This is like the way a casino works. They want lots and lots of small bets, because there is no realistic scenario in which they will be in trouble (some will win, some will lose, and the casino can assume that the money coming in will be enough to cover the money going out.) What they do not like is one huge bet, which may mean that they have to lay out a lot of money all at once. That is why they have limits, and restrict access to high-limit bets.
The trick (particularly with SaaS) is going to be to develop models that allow them to spread this load. This is why the market for cloud infrastructure will be separated from that for SaaS (cloud software).. because there will be too much similarity of usage for most SaaS providers, so they will not get economies of scale in the infrastructure.
2 Lydia Leong October 17, 2008 at 1:44 pm
I don’t think that the desire for small slices is necessarily inherent in the business model; I think it’s more a reflection of the available technology.
The irony, in Google’s case, is that they have an inherently massively parallelizable infrastructure, yet Google App Engine appears to not provide users with any facilities to actually take advantage of it. Some of the performance problems that users are experience with GAE are related to the necessary shift of thinking that must occur to effectively utilize Google’s Datastore vs. SQL, but thus far it seems like in reality, GAE apps don’t run in the same way that Google’s own apps do.
It looks like the SaaS companies with the most efficient infrastructure economics, at least in the near term, will be the ones that scale horizontally on thin-slice compute. Mass-scale, Web-native applications have generally scaled horizontally on 2-to-4 CPU servers for the last decade, but most ISVs trying to convert their software to SaaS now haven’t designed along that paradigm.
I think the economics of cloud infrastructure scalability in general are not particularly dependent upon time-shifting, but rather, upon broadly changing burst-capacity needs. It’s more the fungibility of the flash-crowd apps, like Facebook apps that end up needing thousands of instances worth of capacity for just a few days or weeks — the duration of the fad.