Today, the rivals and big data pioneers Cloudera and Hortonworks announced their merger. In our last year’s Hype Cycles, we marked “Big Data” and “Hadoop Distributions” as obsolete before plateau. They are off the hype cycle this year, but I want to preserve the obituary of big data that my colleague Mark Beyer wrote in 2017, because we still get questions what Gartner thinks about the big data fate. Big data became the new normal and now it is just data.
Born in 2005 with the open-source Apache project that leveraged the distributed process commodity clusters developed in the new internet era, big data taught business users that there was a way to process massive amounts of data — if they knew how to program. Over the course of the next ten years, the lessons learned in big data’s short life were that processing and storage benefit from independent design and optimization. In 2017, big data lessons are being incorporated into various other technology solutions and the primary technology providers (the Hadoop distribution vendors) have begun their evolution to provide data management and processing support for all manner of data assets. This is not the end of these vendors — indeed it is the cocooning and re-emergence after metamorphosis of some of these providers as more efficient and effective solutions in the data management. More importantly, they will be part of the new semantic processing delivery market (that separates storage and process by moving data semantics to the processing tier when appropriate). Big data solution providers that have learned the correct lessons will make big data “normal” for their customers. We strongly recommend that “big data strategy” should be replaced with “making big data part of our everyday strategy.” Rest in pieces, 2017 — in all those other technologies where you belong.
For what is live in 2018, please refer to:
Follow Svetlana on Twitter @Sve_Sic
Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.