by Andrew White | July 11, 2012 | Comments Off on Dark Data is like that furniture you have in that Dark Cupboard
I love it when a “big data” hype fanatics say, “there is more and growing ‘dark data’ and it’s the biggest source of competitive advantage and business transformation!”. The implication is that you need to run hot foot to your nearest BI vendor and buy even more BI stuff. Really?
Dark data is the cute name given to all that data an organization gathers that is not part of their day to day operations. It is old stuff, stuff that turned up in the mail that you kept, ‘just in case’. It is data that you didn’t erase, because “it might come in handy some time”. There is, in most cases, more dark data than there is light data (I guess the data your organizations does use every day). Dark data is a storage vendors’ dream. All those archiving specialists and traditional Information Lifecycle vendors who, despite their name, do NOT look after the birth and life of data, but only at the death (and after life) of data, are jumping for joy. And the BI vendors are just behind it since you will need their tools to help mine that dark vault.
But there is a flaw in this hyped argument about dark data. Unless you, the business user, have an idea of what you want to ask of this dark data, there is no point worrying about it. Good ideas don’t just come out of the woodwork, or spring forth from a data mart. Business people have to have an idea, a question, an argument to test, a theory to explore, a posit to push against. Without this, all that dark data reminds of that extra furniture we all have in that dark cupboard at the top of the stairs. You know what I mean. We all keep that stuff, just in case. We only ever add to the stuff. We only ever remove (and trash) it when the cupboard becomes full and we can no longer afford to move house, just to collect more dark data…er…furniture and stuff.
So please, can we become a little more considerate for business value might be created before we turn on the light on all that big, dark, data? It might even be useful to think about this before we spend too much hard earned cash on managing that stuff that no one bothers to look at. Can we ask of ourselves, what kind of business related question or opportunity might possibly be impacted, and in what way, before opening up the cupboard door?
Read Complimentary Relevant Research
Organizing for Big Data Through Better Process and Governance
With big data past the Peak of Inflated Expectations on the Hype Cycle, organizations are addressing next-level challenges and asking,...
View Relevant Webinars
Hadoop and Spark: Understanding Open Source Opportunities and Risks
As companies build foundational data and analytics infrastructure with Spark and Hadoop, the market continues to shift and evolve in...
Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.