Blog post

Three reasons why the Internet of Things (IOT) will dwarf Big Data

By Andrew White | June 01, 2015 | 4 Comments

Internet of ThingsInternet of EverythingInstrumentationDark DataData and Analytics Strategies

I blogged the other day about how I believe IOT will dwarf Big Data.  Over the weekend I was thinking about this more and three reasons why I think this is the case solidified in my mind.  They are as follows:

  1. Customers versus Everything: Big data is fixated on the customer, or more precisely, the consumer.  Whatever the vendor’s say, most of the hype and work related to big data looks at discovering something insightful about consumers.  Just knowing what the ideal promotion should be is one thing.  How will fulfillment be serviced to make sure that insight is capitalized?  Is the analytic worth paying for if you cannot execute?  After all, an analytic can only go so far: it’s the perception of fulfillment that delivers profit.  As I used to say, “Knowing the train is running late is one thing; being able to change your travel plans is quite something else”.
  2. Insight versus Fulfillment: Big data finds the opportunities; IOT demands a closed loop approach.  IOT does not fixate in consumers, or customers. IOT concerns itself with the instrumentation of everything, and anything.  For the ‘everything’ related to consumers big data might be needed.  For ‘everything’ related to fulfillment, big data won’t help.  Other things will be needed such as modern business process improvement and business applications.  So IOT’s focus is much broader.  IOT needs the full cycle of Plan, Do, Check, Act.  Big data does not.
  3. Wisdom of the Crowd versus the Individual: Big data assumes that there is insight just waiting to be discovered in the vast troves of data, specifically dark data.  IOT adds a wholly different focus for which big data is not designed: small data.  By applying modern advanced analytic capabilities to specific temporal streams of data for individual devices (e.g. passenger car status, heart valve condition) individual tailored insight can be determined.  Big data is all about assumptions derived from large scale pattern discovery; small data is all about discovering differences in the rivers that comprise the lake.  Again, IOT is border than big data.

The original blog attracted some comments (thanks all) and one mentioned the security issue and how just that, for IOT, could spawn another significant sector change.

I am not totally sure that these ideas hold up – they feel right though.  What do you think?

The Gartner Blog Network provides an opportunity for Gartner analysts to test ideas and move research forward. Because the content posted by Gartner analysts on this site does not undergo our standard editorial review, all comments or opinions expressed hereunder are those of the individual contributors and do not represent the views of Gartner, Inc. or its management.

Comments are closed


  • Tobias Lindener says:

    I think you’re making a pretty good point. Especially aspect two really highlights a key challenge of IoT. It’s not only about the connection of everything, it’s about streamlining processes based on the availability of a connected world.

    Yet, I think your third point isn’t precise enough. Surely IoT is broader than BigData, essentially because IoT is also able to create the massive amounts of data to use BigData. On the other hand leveraging the rivers of data won’t change the need for massive pools of data for machine learning and “global” analytics (think Google Now, Microsoft Cortana or IBM Watson).

  • The third point is very intriguing. While Big data looks for patterns (insights as you mention here) in the vast amounts of uncorrelated data, IoT can help to focus on individualization. The key challenge still is how to generate new revenue streams from Big data and bringing in individual contexts might be the key. The concept is no longer limited to sensors in traditional sense but also individual behavior, assets and so on

  • Alexander Nerdsky says:

    – Data extracted from electricity usage can, in principle, make a reasonably accurate guess about what appliances you own and how often you use them.
    – Data extracted from cell-towers can, in principle, tell where you visit and how often.
    – Phones equipped with GPS, know where you are, to a high degree of accuracy.

  • I would put things differently. IoT will bring a reason why the average company will need Big Data solutions or in a headline: IoT provokes Big Data adoption.

    The reason is very simple. Have any smart device send a tweet-sized message every 5 seconds (200 bytes). Sell 1 million smart devices. You end up with 3.3 PB every day. Which means every week a company will have to handle the same load Google was handling every day in 2008. The average company is not able to do this so they will need Big Data solutions. IoT is just a combination of different technology innovations coming together: mobile, cheap micro-servers, easy to use micro-controllers, bluetooth low energy, big data, cloud, 4G connectivity, 3D printing, etc.