There are now 15 projects supported by all 5 distributors I track, and several have had new releases since April. Kafka is the newest addition, and I believe the remaining 4-supporter offerings, Mahout and Hue, will remain unsupported by IBM, who has its own alternatives.
So: what is “broadly supported” Hadoop in April 2016? The Apache Hadoop web site still names Hadoop Common, Hadoop Distributed File System(HDFS™), Hadoop YARN and Hadoop MapReduce as the core components, and gives them a common release number. I leave Common out and call that 3 projects. (Apache still lists Cassandra as a “related” project; I do not include it here, since none of the distributions include it.)
The rest of the “projects supported by all 5 distributors” group: Avro, Flume, HBase, Hive, Kafka, Oozie, Parquet, Pig, Solr, Spark, Sqoop and Zookeeper – adds to 15 projects now supported by all. (Note that there is a Sqoop2 project underway which is not compatible with the existing version. It’s not included here – MapR does, however, support it.)Mahout and Hue are the next group – “supported by 4.” That gets us to 17 projects, unchanged from last time.
There are 15 projects supported by two or three distributors, and ‘ll discuss them in my next post. Following that is the “one or none” group – another 30 projects I try to follow as well as I can. This gives us a total of 62 projects in all – a nightmare if you’d actually like to compose a stack for yourself, as I discuss for Gartner clients in Hadoop Project Proliferation Challenges Selection and Support.
As always, the chart below is based on conversations with and/or web documentation from Amazon, Cloudera, Hortonworks, IBM, and MapR. Distributors’ public documentation of supported distribution contents remains variable and sometimes lags actual releases; see Amazon’s page, Hortonworks’ page, IBM’s page and MapR’s page for their details.