By Merv Adrian and Nick Heudecker
In the first post in this series , I looked at the size of revenue streams for RDBMS software and maintenance/support and noted that they amount to $33B, pointing out that pure play Hadoop vendors had a high hill to climb. (I didn’t say so specifically, but in 2014, Gartner estimates that the three leading vendors generated less than $150M.)
In this post, Nick and I turn from Procurement to Plans and examine the buying intentions uncovered in Gartner surveys. We recently described some of our findings in the Gartner research note Survey Analysis: Hadoop Adoption Drivers and Challenges, and since its publication we’ve been inundated by questions about why “Gartner is so negative about Hadoop.” Questions include:
How has Gartner come to this position when other analyst firms are saying the opposite?
The answer is simple: we looked at the data. We received survey responses from 284 participants in the Gartner Research Circle, a sizable research panel of enterprises from less than $500M to global companies in the multibillions, across a representative set of verticals and geographies, and including both Gartner clients and non-clients. 54% of them said they have no plans to invest in Hadoop in the next two years.
We commissioned the survey from our primary research team to supplement what we were hearing from clients during inquiry. (An inquiry is a 30-60 minute phone call between an analyst and a client. It’s essentially how we contextualize our written research.) Combined, the two of us take about 1000 inquiries per year and more than half of them involve Hadoop. Based on these conversations, we simply aren’t seeing the breakneck adoption the overall hype indicates. Clusters over 20 nodes are rare, and production deployments are even more so. This mountain of anecdotal data gave us a sense that Hadoop adoption wasn’t living up to the hype, but turning several hundred conversations into published research is challenging without incredibly dedicated effort – and we still have inquiries to take and research to write. Hence the survey.
Are there positive notes in this report, especially for vendors in the Big Data/Hadoop space? Aren’t companies like Cloudera, Hortonworks and MapR posting YoY growth?
Two observations here:
- The total number of enterprises measures in the hundreds of thousands. To say that our data suggests that 46% of them will be investing in Hadoop or are already hardly describes a tiny market. It’s not a “negative” statement. Only an accurate one – so far.
- The growth of the Hadoop distributors is from a small base – they are now each claiming hundreds of customers. Our data tells us that many companies are using more than one, so some double counting occurs here. Also, there are differences in what constitutes a “customer.” That in no way diminishes their prospects; as we note in the report, “revenue growth for providers would appear to be in moving to larger deployments within their existing customer base,” and there is ample evidence of that. As with the installed bases of DBMSs, new name business is only a part of the opportunity, even at this very early stage of market development.
Do you expect to see the slowdown continue?
No. Part of the story here is in the maturity of the market as measured by last year’s Gartner Hype Cycle shown below. We’ve noted that Hadoop distributions are in the Trough of Disillusionment, a stage all emerging technologies must go through before leveling off into broad deployment, and that is a normal market development. Intent to purchase tracks this cycle and it’s due to accelerate as Hadoop moves to later stages. To get there, objections such as the ones we describe in the document and in other research will need to be overcome – mostly the issue of “I don’t see the value.”
Getting Hadoop past the Trough requires two things: skills and applications. Despite investment in training programs, broad availability of Hadoop administration skills (value enablers) and analytics skills (value creators) is 2-3 years out. Don’t underestimate this uphill climb. Next, applications taking advantage of the Hadoop ecosystem need to emerge. To this point, the work done on Hadoop has effectively been artisanal data management – everything is built by hand. That’s fine if you’ve got the skills, but most enterprises don’t build software – they buy it. Hadoop can be a great place for those applications to reside.
Where do you see the greatest opportunities for emerging vendors in this space?
Both for distribution vendors and for players in adjacent spaces, the opportunities lie in communicating the business value of their technology. As we move to mainstream buyers (and that is another way to read the Hype Cycle – broad adoption happens after the early adopters are in – and they are now), the buyers are not after technology, but business value. They think differently, and they buy differently.
Moreover, these later stage prospects are looking for “enterprise class” features: fit and finish, ease of deployment, use and management, security, governance…. Opportunities abound, and Hadoop continues with the economic refactoring of the data asset. It’s no accident the big players are all in the game. In fact, their presence will have a big impact on the plans of prospects. The job of Sales begins after the first No.