From MDM to Big Data – From truth to trust

My colleague Ted Friedman was quoted in a SearchDataManagement article today.  I had to blog on this topic of course – he was being quoted on the connection between MDM and Big Data. 

The article (Enterprise master data management and big data: A well-matched pair?) correctly calls out how MDM programs tend to focus on single view of (master) data that drives or sits at the center of transaction data throughout the organization.  Yes, MDM does help but master data is not (let’s not forget) the same as transaction data; and there is a lot of other data around master data used by specific applications.  Too many organizations misunderstand “master data” (Wikipedia does not help) and end up stuffing their MDM hubs will manner of data that should not be in there.  See my Three Rings of Information Governance for an explanation.

Ted nicely captures the point that big data contains a lot of unknowns and of course unstructured or semi structured data (content- the name we use for that stuff) that needs to be made well structured in order for users to understand it.  MDM is more oriented around internal, enterprise-centric data; in an environment the organization feels it has a chance to effect change, and so formal information governance.  MDM is thus a program centered on the concept of “truth”.  If we can’t figure out ourselves what a customer is, we might as well give up and go home.

Big data, and specifically social data, is very, very different to most enterprise data.  And here the article fell short – presumably since the thinking “out there” in the cited analyst community has not gotten very far yet.  There is some dialog in the article about pushing MDM “up against” structured data, to help link or relate the vast set of unknown insight to the known, and this is logical. 

But in our view, that developed a year or so ago, the shift in focus moves aware from “truth” towards an emphasis on “trust”.  I finally managed to publish a note on this recently: Big Data Governance From Truth to Trust.  And this is where data quality tools and projects and information governance programs like MDM, today, start to fail.  They are not geared to help an organization document, understand, manage and govern “trust” in someone else’s data – and nor are they yet ready to operationalize this new “policy” within the organization in order to reduce business risk and maximize business gain, from use of big or social data.

At least this seems to be a twist on an old problem – that offers some rich rewards for those that can exploit it first.