Gartner Blog Network


What’s Next for Big Data?

by Svetlana Sicular  |  December 23, 2014  |  6 Comments

Shortly after joining Gartner, I noticed subtle magic in the air. Not Magic Quadrants – they were too obvious. It was I told you so repeated by many analysts on many occasions. It seemed almost mystical – how could they know? At some point, I caught myself saying I told you so too — it became natural after seeing and researching so much. For example, I talked about personal analytics, multidisciplinary teams instead of a single data scientist, and about Hadoop being a live archive — these came to fruition and became common place. I told you so. (Mom, I hate when you say it.)

☝

In the big data field research back in 2012, we saw that there was a big data maturity gap. It needed a couple of years to close. I told you so. Glad to report, 2014 was the first year when enterprises became serious about big data: They started asking questions beyond “how do I begin my big data initiative?” or “how do I select a Hadoop distribution?” Hortonworks even went public, the first Hadoop vendor to do so. My colleague Merv Adrian went into great depth on the Hortonworks IPO in his blog post Hortonworks IPO – Why Now?

The question is — what’s next for big data? First of all, big data will become the new normal sometime between 2016 and 2018. My colleagues Donald Feinberg and Mark Beyer will say (with well-earned pride and a flair of mysticism), I told you so.

Organizations are finally ready for big data in the cloud. In the second half of 2014, my clients started asking about a data warehouse and Hadoop in the cloud. I am an analyst in Gartner for Technical Professionals — 90% of our clients are practitioners who are doing things right now. Therefore, I anticipate many interesting developments around big data in the cloud soon.

In 2015, I expect a plethora of big data applications on top of data platforms (remember, these platforms already demonstrate acceptable maturity). Big data applications will be mostly analytical, and they will be small, in the “app store” style, with few customizations — that makes support and maintenance relatively easy. People would be able to download big data apps they need and use them like Lego blocks to make their own customizations. Big data apps will put a process or a workflow into the spotlight.

If big data apps proliferate, they will need… data. This means a focus on data governance, data preparation and an ability to painlessly load data into big data stores. This also means self-service, or rather SELF-SERVICE. It would be a bigger and bigger subject from the data management and analytics perspectives. And from the process perspective too, I already told you so.

People are impatient. Those who want self-service are demanding increasingly real-time data access, response and gratification. This will lead to in-memory and streaming advances, but I don’t think it will be next: for practitioners, it will be next after next.

The Internet of Things (IoT) is at the peak of inflated expectations of the hype cycle. Organizations collect more data than they can process: for them, it’s still not just about finding the needle, but getting the hay in the stacks. The Internet of Data sustains the IoT. Companies are collecting new data, asking about new external data sources and searching for dark data within. Many new data sources are personal data – privacy and ethics accompany them. This year, I wrote Maverick* Research: Put Your Data in the Bank, Get Dividends, where I foresee intermediaries — personal data banks — representing individuals. A personal data bank will keep deposited data and multiply its wealth through commercialized digital markets. The Internet of Data and personal data banks are not reality yet, but I am sure soon I would be able to say about them I told you so.

 

The Internet of Data sustains all other Internets

The Internet of Data sustains all other Internets

 

Follow Svetlana on Twitter @Sve_Sic

Additional Resources

Category: data-scientist  analytics  data  data-and-analytics-strategies  data-paprazzi  hadoop  information-everywhere  innovation  inquire-within  

Tags: analytics  big-data  data-paprazzi  end-users  hadoop  hadoop-distribution  information-everywhere  market-analysis  

Svetlana Sicular
Research VP
6 years at Gartner
23 years IT industry

Svetlana Sicular is passionate about bringing analytics to domain experts and helping organizations successfully compete by applying their business acumen in analytics and data science. She is convinced that domain expertise and high-value data are the greatest assets that companies should monetize in new analytics applications. Read Full Bio


Thoughts on What’s Next for Big Data?


  1. As a veteran of the BI space, I have mined the ebb and flow of impatience relative to data access. Twenty five years ago the data was kept in departments and reported on by the business. This meant there was no streaming or machine generated data available as it wasn’t accessible to the business. It doesn’t mean it didn’t exist. It did, and no one was doing anything with it except the odd report if something failed or needed attention.

    Today there is a vibrancy about the need for real time data, but the preponderance of the data consumption is still periodic and analyzed off-line. As the two segments catch up to each other – data and people – they will find that it is not about the data scientists, but about the everyday people who know their core focus area and are not ready to analyze the IoT and IoD. They are focused on the IoP and productivity in real time; they are making micro-decisions that improve business processes. They don’t read graphs. They don’t perform calculations. They know thresholds and target KPIs. They don’t need BI, they need a performance monitoring system that enables them to make decisions – based on visual cues. Enabling a worker to visualization a process and visual function cues to help them use all the relevant data to improve performance before it becomes a problem.

    VisualCue is the first product to market allowing businesses to include a class of data consumer in the decision-making process who was previously completely overlooked. Bringing the capacity to visualize more than 5 metrics simultaneously to an hourly worker who can react in real time is a performance breakthrough. This heralds a new level of data consumption in the enterprise and will be the inflection point of data understanding at all levels for context-sensitive decision-making. Finally, an “I told you so” about who the data consumer really is – everybody.

  2. This is so true and predicted in our book; read inside the book!! http://tinyurl.com/lz45vxn

  3. […] For example, Svetlana Sicular, a researcher in data management and frequent contributor to Gartner, mentions the culture of congratulatory back-slapping that surely takes place this time of year. Or look at […]

  4. Context is the essence of the Internet of Analytics. This will be the glue that binds People, Ideas, Data, and Things into the fabric of communities and will be at the core of individuation.



Comments are closed

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.