Gartner Blog Network


Dark Data is like that furniture you have in that Dark Cupboard

by Andrew White  |  July 11, 2012  |  Comments Off

I love it when a “big data” hype fanatics say, “there is more and growing ‘dark data’ and it’s the biggest source of competitive advantage and business transformation!”.  The implication is that you need to run hot foot to your nearest BI vendor and buy even more BI stuff.  Really?

Dark data is the cute name given to all that data an organization gathers that is not part of their day to day operations.  It is old stuff, stuff that turned up in the mail that you kept, ‘just in case’.  It is data that you didn’t erase, because “it might come in handy some time”.  There is, in most cases, more dark data than there is light data (I guess the data your organizations does use every day).  Dark data is a storage vendors’ dream.  All those archiving specialists and traditional Information Lifecycle vendors who, despite their name, do NOT look after the birth and life of data, but only at the death (and after life) of data, are jumping for joy.  And the BI vendors are just behind it since you will need their tools to help mine that dark vault.

But there is a flaw in this hyped argument about dark data.  Unless you, the business user, have an idea of what you want to ask of this dark data, there is no point worrying about it.  Good ideas don’t just come out of the woodwork, or spring forth from a data mart.  Business people have to have an idea, a question, an argument to test, a theory to explore, a posit to push against.  Without this, all that dark data reminds of that extra furniture we all have in that dark cupboard at the top of the stairs.  You know what I mean.  We all keep that stuff, just in case.  We only ever add to the stuff.  We only ever remove (and trash) it when the cupboard becomes full and we can no longer afford to move house, just to collect more dark data…er…furniture and stuff.

So please, can we become a little more considerate for business value might be created before we turn on the light on all that big, dark, data?  It might even be useful to think about this before we spend too much hard earned cash on managing that stuff that no one bothers to look at.  Can we ask of ourselves, what kind of business related question or opportunity might possibly be impacted, and in what way, before opening up the cupboard door?

Category: big-data  business-intelligence  dark-data  information-lifecycle-management-ilm  

Andrew White
Research VP
8 years at Gartner
22 years IT industry

Andrew White is a research vice president and agenda manager for MDM and Analytics at Gartner. His main research focus is master data management (MDM) and the drill-down topic of creating the "single view of the product" using MDM of product data. He was co-chair… Read Full Bio




Comments are closed

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.