The leading 2017 story of Hadoop distributions is that nobody seems to want to be accused of being in the business of providing them. Some former champions are expanding their shiny new positioning: Cloudera is selling Enterprise Data Hubs and Analytic DBs; Hortonworks offers DataPlanes and Next-Gen Data Platforms; MapR touts the Converged Data Platform. In the cloud world, Amazon’s EMR is at least designed to “run and scale Apache Hadoop, Spark, HBase, Presto, Hive, and other Big Data Frameworks” while on Google’s Cloud Platform page the word Hadoop appears once inside the description of Cloud Dataproc.
I’ve talked in Gartner research about the “disaggregation of the stack” – meaning the use of some of the components but not all, combined with other Apache and non-Apache pieces, by organizations that want to compose their own stack rather than buying someone else’s. (Gartner clients can find that in our Hype Cycle for Data Management.) The former Distributors are doing the same, adding products based on other Apache projects or their own pieces to generate additional revenue streams, partnering with third parties to create go to market offerings targeted at specific use cases, and in some cases, getting out of the business entirely. Since my March blog post, IBM has become the latest to get out of the distro business, dropping the IBM Open Platform and inking a deal with Hortonworks that bodes well for the latter’s strategy built around Atlas (which IBM will also sell.) Of course the providers still sell Hadoop distributions, but as in any maturing market, they need to broaden their story and focus on business outcomes to meet mainstream buyers where they live. Its normal, and it’s appropriate – and hopefully remunerative, since they are still losing money.
The tracker continues to be relatively useful, I hope, for those getting their first projects together – it tells you which versions of Apache projects are in the Hadoop distributions named. All the vendors have other bits the others do not support, and therein lies the competitive differentiation they seek. In some early posts next year, I hope to discuss some of the many other Projects seeing investment and buyer interest. Have a happy New Year, and keep watching.
Edited December 29 for AWS release