On May 17, 2011, HP, IBM, Intel and Red Hat (as governing members) joined BMC Software, Eucalyptus Systems and Suse to announce the “Open Virtualization Alliance”, or OVA (which means “eggs” in Latin, right?). Their stated purposes include “increase overall awareness” of KVM, “accelerate the emergence of an ecosystem” around KVM, and so on.

Sure, the server virtualization market is in dire need of good competition, no doubt about that. In fact, it needed competition ten years ago.

So what’s wrong with open source? Nothing! Xen was introduced in 2003, a mere two years after VMware introduced ESX Server. Xen is widely used – especially by service providers (such as Amazon’s EC2). Citrix XenServer and Oracle VM are based on open source Xen. Wait a minute – this alliance isn’t about Xen, it’s about KVM, right?

That’s concern number one. I have no issues with KVM – except it’s very late to the market. What KVM is really, really good at is what was really interesting a few years ago, both to enterprises and service providers. What they aren’t so good at – ready-made and rich management and automation tools – is what customers need today (and service providers want to tap into an installed base of enterprise customers). So, “accelerating the emergence of an ecosystem” to me is a sad place to start today in a market that has been growing and evolving rapidly over the past ten years. Especially because this alliance helps to further fragment the open source response to VMware. Is VMware cheering this on?

No doubt, this little hypervisor concept has launched a huge trend toward infrastructure modernization, private and hybrid cloud computing. And HP and IBM have been somewhat on the outside looking in. Yes, they missed having a leadership role in a critical trend, and it is a dangerous one to miss, given it’s viral and mutating nature in all things infrastructure.

So, what do we make of OVA? Back to the egg reference – Mark Twain said “Noise proves nothing. Often a hen who has merely laid an egg cackles as if she laid an asteroid.”

Marketing and alliances and rhetorical use of “open” and “standard” all prove nothing. Let’s see some execution, some fire, some innovation. Show me a sense of urgency, some leadership. Not just about hypervisors and hypervisor ecosystems, and not just about catching up – but leaping ahead. Show me a rocket, and prove that there’s an asteroid out there.

At Gartner, we spend a lot of time trying to understand future scenarios, the likelihood of each, indicators that a scenario is likely to occur, impacts on our clients, and what our clients should do. We’ve studied the impact of virtualization on the server market since virtualization was first introduced <begin chest-thumping>and Gartner was the first firm to point out the negative ramifications of virtualization on server volumes<end chest-thumping>. But we’re getting to the moment of truth.

With the exception of the economic collapse in 2009, server volumes have been dependably growing for years. However, virtualization rates are hitting a point that the negative effect of virtualization on the server market are becoming unmistakable. Not in five years. Now.

2010 was a good year for servers – nearly 9 million were sold. My contention is that if virtualization didn’t exist, there would have been 13, or 14, or 15 million sold.

The engine of server market growth has been the growth of workloads. Since 2004, the compound annual growth rate (CAGR) in workloads has been about 16 percent. 2010 was certainly a much better year than that – but if you factor in the the volume decline in 2009, the growth in 2010 just exactly made up the difference.

If the workload CAGR remains steady, server volumes will start to decline in 2011, and we won’t see 2010’s volumes again in this decade.

The good thing – virtualization (and cloud computing) makes it easier and faster to deploy a workload, and that has a tendency to increase the workload CAGR. However, even accounting for faster workload growth, 2010 is either at or near the peak of server volumes for the next ten years.

However, if Moore’s Law is going to be driven by increasing amounts of cores, those cores are going to need VMs to leverage them. Multicore is going to drive higher virtualization densities, and even fewer servers.

What will it take to drive server volumes up? Low virtualization growth, high workload growth, low virtualization densities. A combination of factors that seems unlikely.

Bottom line – there are a number of realistic scenarios for server volumes in the next decade. Each scenario will drive different vendor behavior (and results), pricing, and end user strategies. But – anyone want to place a bet? I’m blogging it, so I’m placing mine right now.

• Whatever happened to the market where there were distinct servers, storage, and networks? Fabric is blurring that.
• What the heck is an operating system any more, and what does it matter when I have a virtual pool of distributed resources I need to use?
• Whatever happened to the boundary between consumer technology and enterprise technology? Consumerization of IT. And not just personal technology devices – some IT services are given away for free (and subsidized by advertising). Which leads to boundaries disappearing in business models.
• Whatever happened to the boundary between outsourcing and insourcing? Now we have cloud computing: public, private, hybrid, and every other variation. Looking for a black and white definition of cloud computing? A waste of time – it’s gray!
• What about ownership of intellectual property? Open source, community collaboration. Is it plagiarism if you add value to existing content? In a society of information, can you afford not to build on what’s already out there? What should 21st century students do?
• What about the boundary between trusted enterprise data and untrusted data? Can we really afford to ignore any business information that might be useful? Isn’t it about what we do with the data, rather than whether the data is 100% trusted and owned by the enterprise? The boundaries of data used for business intelligence have been blown completely down. For that matter, we are entering a period of data overload – some we can trust, some we partially trust, some that is impartial, some that is partial. Successful people and businesses will be able to find value in that data. Unsuccessful people and businesses will drown in the data, or hide from it.
• Whatever happened to the boundary between IT and the business? In some cases, being solidified in the form of services-orientation (e.g., cloud computing), in other cases, the boundary simply does not exist. How many business people can afford to be laggards in leveraging the latest IT capabilities? How many IT personnel can ignore business strategy?
• What about the boundary between applications and operations – and security, for that matter? It used to be that developers threw their creations over the wall for operations to run, with a kiss “good luck”. New applications are being written based on operational models, with automated deployment/operations/optimization in mind. Security is being captured as policy that moves with the application.

Virtualization. Consumerization. Cloud. Instant connections and collaboration. I could go on.

An overall IT megatrend today is a complete and utter blurring of boundaries – which we could handle conceptually, but it directly affects people and market competition. It’s a lot harder to re-skill, re-organize, and react to partners that become competitors and competitors that become partners and partners who are also competitors depending on the situation.

If there is one “skill” that is critical for an enterprise to have, and for individuals to have who use and/or help deliver IT capabilities (which, by the way, is everyone) – it’s “agility.” If you depend on the predictability of competition, and the predictability of a job category, you’re not gonna make it. You or your company will become noncompetitive faster than you can say “blur.”

To use Neil MacDonald’s perfect phrase, success requires “Embracing the Blur.”

(By the way, Neil has pointed out an interesting book by Stan Davis, called – not surprisingly – “Blur.” I need to take a look!)

My first presentation at Symposium 2010 was “

I started the presentation with some fundamental changes in server virtualization since I presented a year ago.

1) Virtual machine penetration has increased 50% in the last year. We believe that nearly 30% of all workloads running on x86 architecture servers are now running on virtual machines.

2) Midsized enterprises rule. For the first time, the penetration of virtualization in midsized enterprises (100-999 employees) now exceeds that of the global 1000 (or it will before year-end). There has been a HUGE uptake in the last year. Also, unlike large enterprises, midsized enterprises tend to deploy all at once – with outside help.

3) Hyper-V is under-performing. Maybe my expectations were too high, but Hyper-V has not grabbed as much market share as I was predicting. I especially thought that Microsoft would be the big beneficiary of midmarket virtualization. Surveys show otherwise – VMware is doing pretty well there. Here’s a theory. Clients repeatedly told us that live migration was a big hole in Microsoft’s offering – even for midmarket customers (to reduce planned downtime managing the parent OS). Microsoft’s Hyper-V R2 (with live migration) came out 8/2009. Was that too late? Did the economy put pressure on midsized enterprises to virtualize early, before Hyper-V R2 was proven in the market? Or did VMware just have too much mindshare?

VMware’s competition is growing (especially Microsoft, Citrix and Oracle), but VMware is still capturing plenty of new customers.

4) Private clouds are the buzz. Every major vendor on the planet who sells infrastructure stuff has a private cloud story today. In the last year, the marketing, product announcements and acquisitions have been mind-numbing. Some of this is clearly cloudwashing (“old stuff, new name”), but we’ve seen a number of smart start-ups captured by big vendors, and important product rollouts (notably VMware’s vCloud Director). Now the question is – what will the market buy?

5) IaaS Providers Shifting to Commercial VMs. IaaS (infrastructure as a service) providers have focused on open source and internal technologies to deliver solutions at the lowest possible cost. But that’s changing. In the past year, there’s been a rapidly growing trend for IaaS providers to add support for major commercial VM formats – especially VMware, but also Hyper-V and XenServer. The reason? To create an easy on-ramp for enterprises. As enteprises virtualize (and in many cases, build private clouds), the IaaS providers know that they need to make interoperability, hybrid, overdrafting, migration as easy as possible. The question is whether that will require commercial offerings (such as VMware’s vCloud Datacenter Services, or Microsoft Dynamic Datacenter Alliance), or if conversion tools will be good enough. I tend to think that service providers better make the off-premises experience as identical to the on-premises experience as possible – and I’m not sure conversion will get them there.

Gartner’s Symposium this year is a blow-out – more than 7,500 attendees, and more than 1,600 CIOs. That means a very busy week of presentations and one-on-ones. As an analyst, what I always find interesting is “the buzz”. You get a real good sense of what’s hot based on one-on-one load, and one-on-one topics. I was one of a few analysts fully booked a few weeks before Symposium, so my topics are hot. The questions? Continued interest in virtualization, but shifting heavily to cloud computing, both private and public.

Because of presentations, roundtables and so forth, I only had 35 one-on-one slots available. 11 of those are on virtualization (mostly VMware and Microsoft). 9 are about cloud computing (mainly what’s ready, which services, which providers, customer experiences). 14 are about private cloud (how do I start, VMware’s vCloud, etc.).

The sense I get so far is the interest in cloud computing continues to grow, but there is more real activity and near-term spending on private cloud solutions. A lot of interest in VMware’s vCloud – but attendees want some proof first.

At the end of the week, I’ll summarize what I learned. Should be a great week!

Private cloud computing is rapidly moving up the Gartner hype cycle. In terms of raw market hype, I think we’ll peak late this year. VMware’s “Redwood” won’t be the only announcement – every major infrastructure vendor in the planet will likely put “private cloud” in their announcements, their marketing, their product names.

So before we get too overwhelmed with private cloud computing mania, what’s going to be real, and what isn’t? How will private cloud computing be used?

Just like early virtualization deployments, development and test is the favorite starting point for private cloud computing. Take out the middle-man, and provide a self-service portal for developers to acquire resources. Manage the life cycle of those resources, and return them to the pool when the developer is done. Dev/test is a perfect starting point, because there is a need for rapid provisioning and de-provisioning.

What’s next?

I think the next logical place will be the computing sandbox. This is a place for production workloads that need to be put up quickly – a stand-alone web server, a short-running computational task, a pilot project. “I need it NOW.”

The sandbox will especially be the place to put a workload prior to full production deployment internally, but when it needs to go up fast – and when external deployment (in the “public cloud”) isn’t appropriate for one reason or another.

Sandboxes can have different operational rules than normal production workloads. For example, perhaps it is a short-term “lease” and expires after thirty days. Perhaps the software is never maintained or patched during that window. Perhaps there is no backup or disaster recovery in place for those workloads. Perhaps security coverage is limited.

While a workload is running in a sandbox, the administrivia required to get appropriate approvals and fulfill organizational process requirements can be finished in parallel.

Ideally, after some period of time (like at the end of a thirty day lease), there might be a way to move the workload from the sandbox to full production, with all of the service level requirements in place.

Many large organizations will start with dev/test first, and build a sandbox next. I believe for many organizations the sandbox itself will mature and become a broader and more capable private cloud service. But there’s no rush.

The fact is, not every IT organization needs a fully self-service interface, and many smaller organizations see no value in usage metering. They simply want to deliver services faster. For them, a 70% private cloud is absolutely good enough.

There is still value in virtualizing your resources, automating how the resources are allocated to meet demand, automating provisioning based on standard service offerings in a published service catalog. But you may want a person in the middle of the process. Or you may want to route the pure self-service requirements to your favorite external cloud provider rather than build your own. And that’s OK. It all comes down to business requirements, return on investment, and future strategy (including the potential to evolve to external cloud providers in the future). How far you go is your decision. 

So while most enterprises may consider private cloud their goal, and vendor hype is going to skyrocket on how to reach that goal – my bet is that most organizations will find that a less than pure private cloud is going to be good enough.

1) In my presentation on server virtualization on Monday, before I started, I asked the audience how many of them considered private cloud computing to be a core strategy of theirs. 75% raised their hands (I expected maybe one-third). Clearly, everyone has a different idea of what private cloud computing means (or doesn’t), but the fact that so many people have glommed onto the term is very interesting.

I described the three most common things that were being described as private clouds:

• IT defending its turf: Shared services that were being re-labelled as private clouds (but without a self-service interface, or much automation at all)
• Vendors defending their products: Old products being re-labelled as private clouds in a box (I described most of these as “lipstick on a pig”)
• Advanced server virtualization deployments: Although few have a true self-service interface, the intention is certainly there

2) My one-on-ones shifted heavily from virtualization toward cloud computing and private cloud computing. I had 18 one-on-ones that discussed server virtualization, and 26 that discussed cloud and private cloud.

3) Several one-on-ones were very interesting: one company with more than 5,000 VMs heading toward 10,000; maybe a dozen that were focused entirely on taking their leading-edge virtualization infrastructure into private cloud; three companies considering discussions with Terremark on possibly licensing their service interface and usage metering software (they know VMware will probably deliver something in a year, but don’t want to wait); several users wondering what Microsoft’s virtualization strategy was, and especially whether they were focused on private cloud computing as a next step; one that questioned VMware’s vendor maturity with respect to customer relationships (felt that Microsoft did an excellent job with TAMs and so forth to truly work with the customer as a trusted advisor – while VMware’s TAMs were more like communications conduits).

4) In my “debate” with analyst Eric Knipp (on the topic of public vs. private cloud computing), we took a vote at the beginning and end of the session to test audience opinions on the subject. At the beginning, about 15% felt that public cloud computing would “win” (the question was intentionally vague to let the attendee decide what “win” meant). About two-thirds though that private cloud computing would win. By the end of the debate, perhaps a few more thought public cloud would win, about 40% thought private cloud, and about 40% voted otherwise (which included a hybrid model). I suppose the surprise to me (yet again – do I never learn?) was the overall momentum in the concept of private cloud.

The challenge with private cloud computing, of course, is to dispel the vendor hype and the IT protectionism that is hiding there, and to ensure the concept is being used in the right way – as a stepping-stone to public cloud, based on a timing window, the lack of a mature public cloud alternative and a good business case to invest internally.

Gartner analysts spend tremendous time and effort preparing for this conference. Presentations are due to our editing and multimedia departments weeks in advance, but for some reason, IT doesn’t stop, so there are very few analysts who turn in their work on time. Personally, I always make changes as late as the day of the presentation – that’s just a little more real-world.

I love these conferences. During the year, I spend a large percentage of my time on the phone with clients (600 or so calls this year?). I also visit with clients face-to-face throughout the year (I think I visited with perhaps a hundred this year). However, nothing compares with the density of client conversations that take place at Symposium.

For me, Symposium is about four days of constant client interaction. This year, I’ll deliver two presentations (one on cloud and private cloud computing, one on virtualization), a debate (is private cloud real?), a client roundtable, about 40 one-on-ones, two breakfasts with clients, two lunches with clients, a dinner with one client, and another dinner with a few dozen key CIOs. History says, all remaining open time will disappear as soon as I arrive. This will be solid 7am to 10pm client discussions.

I know they come to get advice from us, but we come to hear about their issues.

There is no better way to measure the pulse of IT users. Every interaction is a specific issue that a decision-maker needs to solve. I’m put on the spot, every thirty minutes or so. Sometimes, I can’t answer a question, but I can usually move them closer to an answer, or line up another Gartner analyst who can. But most of the time, there’s a two-way transfer that takes place – I give the client actionable advice based on 25 years of experience and thousands of real client interactions, and they give me a new and up-to-date real-world issue.

An analyst’s job is about being an expert who can help clients with actionable and strategic advice. But an analyst’s job is also about being a learning machine. There’s a tremendous amount of information out there, and it’s our job to find the important nuggets, filter out the chaff, find the patterns, and determine the actions that help our clients the most. And this is what makes the job interesting and keeps me on top of my game.

This year I’ll be active tweeting during the conference. Of course, nothing confidential about individual clients, but I’ll tweet about the pulse of the market and things that are coming up often (tombitt on Twitter, and I’ll hash my tweets with #GartnerSym).

And for those of you coming to the conference, I’ll see you there, and hope you get as much out of it as I do!

The midmarket is waking up. At Gartner, the number of midmarket clients who are just starting out with virtualization has been growing exponentially for the last year or so. We just completed a worldwide survey of nearly 1,500 companies with 100-999 employees. What you see is pretty compelling evidence that a market that nearly ignored server virtualization for years started to wake up about two years ago, and is rapidly becoming a dominant opportunity for vendors.

Furthermore, it’s clear that midmarket enterprises virtualize at a different rate than large enterprises. Large enterprises virtualize over a long period of time, at the rate of hardware replacement. Midmarket enterprises tend to virtualize as a part of a single project, or over a 1-2 year period. 

We predict that midmarket enterprises will have a higher percentage of servers virtualized by year-end 2010 than the global 500 – who have been virtualizing since 2001. That means 2009 and 2010 will be huge.

What does this mean? I think it means that VMware messed up, and effectively ignored the midmarket (especially in terms of entry price) until Microsoft came to the table. I think the price war is going to take a serious toll on vendors at the low-end. I think midmarket software vendors who have been able to ignore virtualization are going to have to make changes very quickly – in terms of support, but also pricing and licensing. I also think that we are going to have some pretty messed up midmarket enterprises – who virtualize too fast, use immature technology, lose some control of their environment, open new security gaps, etc.

Gonna be a bumpy ride!