Gartner Blog Network


Two Main Questions about Big Data

by Svetlana Sicular  |  April 16, 2015  |  1 Comment

More and more people describe the speed of big data technologies in dog years. In the lightning-fast cadence (“fail fast”), considerations for actual technical professionals who are deploying big data solutions fall into cracks. Vendors look at abstract “customers.” Big data ecosystem talks about “data scientists.” A new term “data engineer” is bubbling to the surfaces of data lakes. And real people — information architects, software engineers, managers of analytics teams — are coping with uncertainty. They don’t know about each others’ uncertainty; they think that they are dealing alone with rapid changes, brittle solutions and permanent immaturity brought by a yet one more promising technology. And they all ask the same two questions:

  • How do we bridge our information silos?
  • How do we get to the data faster?

In essence, two main questions are about data availability. In some cases, big data technologies are the answer, and in some cases, they are not. It depends… on governance. Big data technologies are capable of solving both questions when they are implemented under governance that supports new solutions.

Gartner actively promotes the idea of bimodal IT. Mode 1 (“marathon”) embodies stability and long-term planning; Mode 2 (“sprint”) embodies agility and quick results. Bimodality is necessary to survive in the digital economy. Often, albeit not always, the traditional and big data technologies are divided along the lines of marathon and sprint. The “sprint” mode can address both key questions — bridging information silos and getting to the data faster — when companies implementing new technologies consciously plan bimodality.

Let’s take an example of the EDW and Hadoop.

BiModalSometimes, the results are achieved in Hadoop because of pure technology, but sometimes because Hadoop is not as strictly governed as the EDW. And it is okay to let people loose in Hadoop (or any other technology for that matter) as long as you are conscious of the differences in governance, and moreover, as long as you know how to switch information governance from sprint to marathon when your new solutions become mission-critical.

I wrote a new research note Upgrade the Enterprise Data Warehouse Architecture With Hadoop for actual technical professionals who are asking these two questions:

  • How do we bridge our information silos?
  • How do we get to the data faster?

In the last question, “faster” often means “sooner.”

 

Follow Svetlana on Twitter @Sve_Sic

Category: data-scientist  analytics  big-data  data  data-paprazzi  hadoop  information-everywhere  innovation  inquire-within  

Tags: data-scientist  analytics  big-data  data-paprazzi  end-users  hadoop  information-everywhere  

Svetlana Sicular
Research VP
6 years at Gartner
23 years IT industry

Svetlana Sicular is passionate about bringing analytics to domain experts and helping organizations successfully compete by applying their business acumen in analytics and data science. She is convinced that domain expertise and high-value data are the greatest assets that companies should monetize in new analytics applications. Read Full Bio


Thoughts on Two Main Questions about Big Data


  1. […] Two Main Questions about Big Data “More and more people describe the speed of big data technologies in dog years. In the lightning-fast cadence (“fail fast”), considerations for actual technical professionals who are deploying big data solutions fall into cracks. Vendors look at abstract “customers.” Big data ecosystem talks about “data scientists.” A new term “data engineer” is bubbling to the surfaces of data lakes. And real people — information architects, software engineers, managers of analytics teams — are coping with uncertainty. They don’t know about each others’ uncertainty; they think that they are dealing alone with rapid changes, brittle solutions and permanent immaturity brought by a yet one more promising technology. And they all ask the same two questions…” Via Svetlana Sicular, Gartner […]



Comments are closed

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.