by Svetlana Sicular | June 23, 2015 | Comments Off on Big Data Is Pregnant with Analytics
We are at the interesting point: big data time is over. It is now big data analytics time. Many organizations are at the point when they have figured out how to get data in Hadoop (or other big data stores), but not — how to get the data out and derive value from it. These companies are becoming increasingly nervous under the pressure of rapidly growing amounts of unprocessed data (a.k.a. WRITE ONLY data that nobody will ever read).
The convergence of cloud, mobility and social computing that started several years ago culminated now in the first truly widespread big data analytics use case — the Internet of Things, IoT. Companies in very different industries — insurance, oil and gas, healthcare, transportation and agriculture among many others — are deploying sensors and collecting data, generated by them.
The rise of data lakes reflects the nature of the current point in time. Data lakes signify uncertainty, when organizations want to store more and more data generated with enormous speed, hoping to make sense of this data someday. Companies need a conventional name for a data storage that allows them to keep their options open for future analysis — this is a data lake. The more data is in the lake, the harder it is to separate the signal from the noise. The signal is there, among a myriad of other signals, go fish.
Analytics is the way out of the data lakes, it will help to find value in big data stores. However, analytics is now different: it is not just a clever tool for analysis, but also the whole architecture to put data in the analytic-ready form. And remember, it is big data analytics — different solutions and algorithms are required at scale.
Moore’s law is still hard at work: for example, server memory is measured now in terabytes compared to gigabytes two years ago, so clusters of severs can keep tens of terabytes in memory — this paves the road to fast in-memory analytics. Apache Spark — a fast, in-memory processing and analytical framework — came to focus at the right time, in the right place: it is leading the shift from big data storage to big data analysis.
In early 2014, I wrote in a blog post, “The rocket ship of big data analytics is launched and on its way to orbit.” Happy to report: the rocket ship of big data analytics reached orbit!
Follow Svetlana on Twitter @Sve_Sic
Category: analytics data-and-analytics-strategies data-paprazzi hadoop information-everywhere
Tags: analysis analytics apache-spark big-data data-lakea data-paprazzi hadoop information-everywhere market-analysis
Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.