Blog post

The Illusions of Big Data

By Svetlana Sicular | July 24, 2013 | 1 Comment

Inquire WithinInformation EverywhereHumansdata paprazziData and Analytics Strategiesdataanalytics"Data Scientist"

I’m afraid that more data and more analytics will create an illusion of solutions while the problems still persist. Many problems are intrinsically not possible to solve. Thousands of years of philosophy testify to that.  Philosophy aside, people cannot agree on elementary things: some of us have firm reasons to vote for democrats, some of us have iron-clad arguments in favor of republicans. And after this, it is ridiculous to expect big data technologies to solve many if not all problems.

Companies are increasingly interested in expanding to predictive and prescriptive analytics – answers will be shifting from deterministic to probabilistic.  But probability itself is an illusion. It’s easy to forget that statistics doesn’t apply to a single person or event because this person or event can be an outlier. Conversely, while an individual usually behaves rationally, an aggregated crowd is irrational.  Making sense of the irrational is the eternal challenge.

When we estimate probabilistic answers, we tend to use the 80/20 rule which works as described by my colleague Mark Beyer who says that 80% of 80/20 rules are garbage.  We assume that 80% probability is good enough for predictive or prescriptive analysis.  Some 80/20 rules even assume that 90% is good, and 80% is not.  In reality, most probabilistic answers have a very low probability of being “correct” (think philosophically): 20% of the 80/20 rule would be a very decent number. And there are simpler mistakes: I’ve just recently seen an example of 24 successes vs. 12 successes resulting in 50% improvement (the example was more complicated because of the accompanying non-related big numbers and complex descriptions that distracted from the essence of 24 vs. 12).

At the last year’s Gartner BI Summit, a panel of the leading BI vendors was asked about a percentage of BI failures.  The first panelist answered 50%, the second, a competitor of the first one, decreased estimated failures to 40% (↑), and the last one, who has an obvious low limit, said that most BI implementations do not fail (↑).  There was a similar vendor panel at this year’s Gartner BI and Analytics Summit: the vendors varied but they got the  same question – about the rate of BI implementations failures.  The first vendor gave the answer of 50%, the second one gave 60% (↓), and so on (↓) until the last vendor who didn’t have much wiggle room said that 99% of BI implementations fail (↓).   Both years, vendors had really good explanations of their opinions. But what drove their answers? Data and analysis? Mostly, pressure, not the actual reasons, which are subjective too. This illustrates an illusion of getting the right answers.

An analytical insight leads to different conclusions depending on a decision maker too.  Some people cannot make a decision and enjoy a process, postponing decisions as long as they can, until circumstances change and the insight gets obsolete.  Some people naturally make decisions quickly and then accept implications, good or bad; often, hasty decisions are not very good. Everyone of us, quick or indecisive, made bad judgments in the presence of all necessary information. As suggested by my colleague Nigel Rayner, machines could make a better choice in certain cases, but they are programmed by people about whom I talked above.  Or what if machines discover something obvious to people?  We have seen many times that the answer is a non-event.

By having more and more, and bigger and bigger data, and by delivering more and more analytics, including the current favorites at the world’s leading whiteboards and blueprints — predictive and prescriptive types — we might conclude that many more problems are already solved.  And this will be just an illusion.  Look!

Optical Illusion
The Optical Illusion

P.S. Don’t give up solving problems though. “You are never given a wish without also being given the power to make it true. You may have to work for it however,” said Richard Bach in Illusions.


Follow Svetlana on Twitter @Sve_Sic

The Gartner Blog Network provides an opportunity for Gartner analysts to test ideas and move research forward. Because the content posted by Gartner analysts on this site does not undergo our standard editorial review, all comments or opinions expressed hereunder are those of the individual contributors and do not represent the views of Gartner, Inc. or its management.

Comments are closed

1 Comment

  • Hans Willems says:

    What an excellent blog post Svetlana. Spot on.

    We want to use big data to make bigger and better decisions. More data automatically deliver better results, isn’t it? Its in our thinking system.

    Whatever we decide, we do it with a purpose. For a company the purpose should be helping individual customers to make the best decision for them and the company.

    In this context, big data could better be used to continuously listen to and understand individual customers and use this information on the fly to support this process. Millions of ongoing micro decisions will guide the individual customer in his journey.

    Don’t throw away the big decisions yet. It would ask for an immediate and full reset of the thinking system. It would only result in another illusion.