There are now 15 projects supported by all 5 distributors I track, and several have had new releases since April. Kafka is the newest addition, and I believe the remaining 4-supporter offerings, Mahout and Hue, will remain unsupported by IBM, who has its own alternatives.
So: what is “broadly supported” Hadoop in April 2016? The Apache Hadoop web site still names Hadoop Common, Hadoop Distributed File System(HDFS™), Hadoop YARN and Hadoop MapReduce as the core components, and gives them a common release number. I leave Common out and call that 3 projects. (Apache still lists Cassandra as a “related” project; I do not include it here, since none of the distributions include it.)
The rest of the “projects supported by all 5 distributors” group: Avro, Flume, HBase, Hive, Kafka, Oozie, Parquet, Pig, Solr, Spark, Sqoop and Zookeeper – adds to 15 projects now supported by all. (Note that there is a Sqoop2 project underway which is not compatible with the existing version. It’s not included here – MapR does, however, support it.)Mahout and Hue are the next group – “supported by 4.” That gets us to 17 projects, unchanged from last time.
There are 15 projects supported by two or three distributors, and ‘ll discuss them in my next post. Following that is the “one or none” group – another 30 projects I try to follow as well as I can. This gives us a total of 62 projects in all – a nightmare if you’d actually like to compose a stack for yourself, as I discuss for Gartner clients in Hadoop Project Proliferation Challenges Selection and Support.
As always, the chart below is based on conversations with and/or web documentation from Amazon, Cloudera, Hortonworks, IBM, and MapR. Distributors’ public documentation of supported distribution contents remains variable and sometimes lags actual releases; see Amazon’s page, Hortonworks’ page, IBM’s page and MapR’s page for their details.
Read Complimentary Relevant Research
Organizing for Big Data Through Better Process and Governance
With big data past the Peak of Inflated Expectations on the Hype Cycle, organizations are addressing next-level challenges and asking,...
View Relevant Webinars
Hadoop and Spark: Understanding Open Source Opportunities and Risks
As companies build foundational data and analytics infrastructure with Spark and Hadoop, the market continues to shift and evolve in...
Category: apache avro cassandra flume hadoop hbase hdfs hive kafka mapreduce oozie apache-parquet pig solr spark sqoop apache-yarn zookeeper big-data cloudera gartner hortonworks hue ibm biginsights mapr open-source
Tags: apache flume hadoop hbase hdfs hive mapreduce oozie pig sqoop yarn zookeeper big-data-2 biginsights cassandra cdh cloudera hortonworks ibm mapr
Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.