The Big Data Obituary

Today, the rivals and big data pioneers Cloudera and Hortonworks announced their merger. In our last year’s Hype Cycles, we marked “Big Data” and “Hadoop Distributions” as obsolete before plateau. They are off the hype cycle this year, but I want to preserve the obituary of big data that my colleague Mark Beyer wrote in 2017, because we still get questions what Gartner thinks about the big data fate.  Big data became the new normal and now it is just data. 

Born in 2005 with the open-source Apache project that leveraged the distributed process commodity clusters developed in the new internet era, big data taught business users that there was a way to process massive amounts of data — if they knew how to program. Over the course of the next ten years, the lessons learned in big data’s short life were that processing and storage benefit from independent design and optimization. In 2017, big data lessons are being incorporated into various other technology solutions and the primary technology providers (the Hadoop distribution vendors) have begun their evolution to provide data management and processing support for all manner of data assets. This is not the end of these vendors — indeed it is the cocooning and re-emergence after metamorphosis of some of these providers as more efficient and effective solutions in the data management. More importantly, they will be part of the new semantic processing delivery market (that separates storage and process by moving data semantics to the processing tier when appropriate). Big data solution providers that have learned the correct lessons will make big data “normal” for their customers. We strongly recommend that “big data strategy” should be replaced with “making big data part of our everyday strategy.” Rest in pieces, 2017 — in all those other technologies where you belong.

For what is live in 2018, please refer to:

Hype Cycle for Information Governance and Master Data Management, 2018

Hype Cycle for Data Management, 2018

Hype Cycle for Artificial Intelligence, 2018

Hype Cycle for Data Science and Machine Learning, 2018

Hype Cycle for Analytics and Business Intelligence, 2018

Follow Svetlana on Twitter @Sve_Sic