The Big Data Obituary

by Svetlana Sicular | October 3, 2018 | 1 Comment

Today, the rivals and big data pioneers Cloudera and Hortonworks announced their merger. In our last year’s Hype Cycles, we marked “Big Data” and “Hadoop Distributions” as obsolete before plateau. They are off the hype cycle this year, but I want to preserve the obituary of big data that my colleague Mark Beyer wrote in 2017, because we still get questions what Gartner thinks about the big data fate. Big data became the new normal and now it is just data.

Born in 2005 with the open-source Apache project that leveraged the distributed process commodity clusters developed in the new internet era, big data taught business users that there was a way to process massive amounts of data — if they knew how to program. Over the course of the next ten years, the lessons learned in big data’s short life were that processing and storage benefit from independent design and optimization. In 2017, big data lessons are being incorporated into various other technology solutions and the primary technology providers (the Hadoop distribution vendors) have begun their evolution to provide data management and processing support for all manner of data assets. This is not the end of these vendors — indeed it is the cocooning and re-emergence after metamorphosis of some of these providers as more efficient and effective solutions in the data management. More importantly, they will be part of the new semantic processing delivery market (that separates storage and process by moving data semantics to the processing tier when appropriate). Big data solution providers that have learned the correct lessons will make big data “normal” for their customers. We strongly recommend that “big data strategy” should be replaced with “making big data part of our everyday strategy.” Rest in pieces, 2017 — in all those other technologies where you belong.

For what is live in 2018, please refer to:

Hype Cycle for Information Governance and Master Data Management, 2018

Hype Cycle for Data Management, 2018

Hype Cycle for Artificial Intelligence, 2018

Hype Cycle for Data Science and Machine Learning, 2018

Hype Cycle for Analytics and Business Intelligence, 2018

Follow Svetlana on Twitter @Sve_Sic

Additional Resources

Category: ai analytics big-data data-paprazzi hadoop

Tags: big-data cloudera hortonworks hype-cycle

Svetlana Sicular
Research VP
6 years at Gartner
23 years IT industry

Svetlana Sicular is passionate about bringing analytics to domain experts and helping organizations successfully compete by applying their business acumen in analytics and data science. She is convinced that domain expertise and high-value data are the greatest assets that companies should monetize in new analytics applications. Read Full Bio

Thoughts on The Big Data Obituary

Nancy says:

October 16, 2018 at 1:13 am

Beautiful to read this article.Good job

Comments are closed

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.

Gartner Blog Network

Svetlana Sicular