“We had an ERP (think large packaged business application) migration program 18 months ago or so, and we decided to add an MDM strategy.  The MDM part fell away and was not followed through though the ERP migration was a success.  However, now we have no formal process for assuring that data in our systems is clean, consistent, and accurate.  Now we have business problems that are surfacing since the data is not consistent and business processes are not producing the behavior or results we expect.”

No kidding.

The choices the client faces are difficult:

• Throw some data quality tools at the data as needed to clean what is dirty, and treat this is a periodic, or series of “one off” exercises to clean up the stuff
• Use an external service provider on-demand to clean and keep clean the data and provide enriched data services
• Build out a solution (people, and process, and technology), that sustains a formal process to assure data consistency and quality (akin to what the MDM program would have done).

The point from the conversation that was most ironic for me was this.  We talked about what happened to the MDM program and the efforts to establish “governance”, and one of the IT folks on the call said, “Yes, the (business) users subverted that effort [and now look at the mess we are all in].” 

I thought this was weird.  Here was “IT” knowing what needed to have been done, criticizing the business for doing the minimum that they needed – at the time.  I would posit that to get ERP up and running, you don’t need to establish a data governance program!  By “running” I mean operating but not for any substantial period of time.  However, the ERP system will not support the business requirements or deliver the value it was supposed to over any period of time if data governance (spanning master data as well as other data inside and outside of ERP) is not stood up.  The business did not “subvert” the program; they probably didn’t think they needed it; and saw no direct value from it.  “ERP” was the goal, and now everyone has to live with the consequences.

Why is it that so many end users find themselves in this situation?

Clearly data quality is applicable to many kinds of data integration and data management routines.  MDM is one such (not sure there is such a thing as ‘data governance’ that sits outside of context, but that’s another idea for a blog) program and Netrics has previously partnered with several MDM vendors to provide matching capability as part of a broader data quality focus.  Netrics focus had been entity resolution and matching, very common in party data oriented MDM programs (think MDM of Customer Data).  I have not seen Netrics play as much a role in MDM programs related to “things” (think MDM of Product Data) though they did sport a very small number of clients in this area.

With Oracle recently acquiring SilverCreek (specialist focus on the product domain) this new acquisition highlights the dwindling set of DQ tools for MDM (and other interested) vendors to partner with, and/or acquire.  The acquisition seems logical, and good, for packaged MDM (TIBCO offers one) though; but as the music dies down, who will be left standing without a partner….

Harnessing core data issues to improve organizational efficiency, profitability and competitive advantage. There is much ado about data quality in business today, and with good reason. Both IT managers and business decision makers alike are awakening not just to the direct costs of poor data quality, but more importantly, to the potential for true competitive advantage that superior data quality can yield.

Specifically, carefully planned and executed data quality and data integration initiatives can be integral to an organization’s ability to gain a 360-degree view of itself, its operations, and its customer base.

Well sure, that’s good stuff.  But what such “data quality and data integration initiative” provides 360 degree view of such things?  Why don’t people just call it out – you mean Master Data Management?  Why do some folks cling to phrases that don’t actually say or mean anything specifically?  Is MDM a data quality initiative?  Sure, I’d go with that.  Is MDM a data integration initiative?  Sure, I’d go with that too.  Is MDM a governance initiative?  Sure – I’d even go with that.  What I would say though is that “data quality”, and “data governance” and “data integration” lack the context I need to link to the business, and hence value the technology or discipline yields.  I wish SAS would say it like it is: Try/start with MDM – it helps a lot of things

“The findings of this report mirror our own client experiences, namely that there is an uplift in MDM adoption and an acceleration in delivery timeframes,” says  Jill Dyche, partner and co-founder of Baseline Consulting. “The fact that data quality, data governance, and data enrichment processes may accompany an MDM initiative make it all the more attractive as an enterprise solution.”

The part that made me smile was the implication that MDM could exist without data quality, data governance, and data enrichment processes.  Our view, and definition of MDM, always implied this.  How else could MDM (the discipline) be achieved?

Perhaps the comment is targeted more at “what is actually going on in the real world that is called MDM” in which case I can understand the point.  But, given the flurry of exchanges in the last 2 days, I thought this article’s timing was exemplarity (and perspective relevant).

• Business users tend, if you are luck, to accept that they “own” process, but few accept that they do, or should, own data.  Conclusion – get over it, and start talking of “stewardship” instead
• When someone talks of “governance”, without any context, its tantamount to talking about fools gold.  Conclusion – governance, out of context, is dangerous; add context in order to make it meaningful
• “Governance of data/information” needs to start, or be applied first, with a focus on data/information that is meaningful to the business: Conclusion – start a governance focus on “enterprise information” (that is, stuff that’s important like master data or reference data).

One question that came up from the vendor was this: does a focus on data quality (DQ) confuse, or complicate, a message related to data governance (DG)?  It turns out that the vendor perceives a risk that users will think that a message/offering oriented around “data quality” might get confused (i.e. might compete) with something oriented around “data governance”. 

Clearly there is a potential for this.  Even in our research, “data quality” at an holistic level, implies a broad view of the quality, integrity, and consistency of data and/or process etc.  So at one level, there is overlap.  However, the reality is that most users do NOT take that holistic view; they equate “DQ” with a technology that is applied to a problem at a point in time.  Few use the term DQ to equate to a living, breading process that manages, or ‘governs’, the process or data. 

Bottom line – DQ and DG are different, but related.  The vendor does run a minor risk (in my view) but I think it’s a slight one.  For sure, there are lots of “DQ” messages and offerings on the market today; and there will be a lot more related to DG soon.  I’d go with a focus on DG and message how DQ technology supports DG.

The article puts it this way:

When a system creates data, and when that data leaves that system, the data should be checked and corrected.  Bad data should be viewed as a hazardous material that should not be transported. The moment you generate data, you have the implicit responsibility to establish its accuracy and integrity.  Distributing good data to your competitors is unacceptable; distributing bad data to your team is irresponsible. And when bad data is ignored, it’s negligence.

I think this article is writing about a big problem but concluding a solution that is incorrect.  They key error (for me) emerges when the article concludes that one application can understand the context by which its output is used by another application.  When did applications (or designers of applications, or users of applications) concern themselves or worry about the context in which their data may or may not be used by any number of other applications, many of which were not designed before the first application was? 

If business applications are developed or deployed one at a time (like how most of the world works) we know for a fact that:

1. data might be perfectly accurate and fit for use (in application A), and
2. this can be of little value or interest for application B if the context changes

Is this a “fault” of the first application?  No, not necessarily.  It is hard to guess what my wife is thinking – even when she is talking to me about what is on her mind.  It’s hard for me to figure out what I am thinking about, half the time.  Is it a design flaw that leads to this issue?  I don’t think so.  I don’t think that 20 years ago we all got together and said, “hey guys, lets design applications in silos so that we confuse each other with different data definitions.”  It just happened – and it happened when we tired explicitly to stop it happening.  It is not a design flaw; it is a weakness in the view of what the design turned out to be.

I do not think that even in 2009 we can ask any application (amongst a peer of applications) to take upon itself the responsibility of “cleaning” up its data before it is shared.  No single application can understand what “clean” means in the context of other application use.  Data might be “100%” accurate – for one purpose – and be totally useless for another.  Why try to force every application to do something for which it was never designed to do?  Why replicate this responsibility across all these siloed applications?

I think we would all agree that context is key – that knowing the context for which information is being used is required, in order to determine if the information is fit for purpose.  But that context has to be supplied at the time of use (i.e. as part of the business applications, service, or inquiry).  Disciplines like MDM standardize and centralizes how master data is governed across the myriad contexts that exists across the firm.  That model seems to work conceptually, and work in the 6 years life cycle of MDM show that it is beginning to work well in practice too. 

Focusing on solving this problem, “one application at a time”, is “business as usual” and the sign of lethargy and inertia.  This approach will waste your money faster than a consulting project. 

However, I love the comment, “Distributing good data to your competitors is unacceptable; distributing bad data to your team is irresponsible.”  So I believe the goal of the writer is valid – just that architecturally the focus on applications (that cause the problem in the first place) to solve a problem that is dependent on context for which the application cannot understand, is false.

Operational MDM centers on assuring “single view” of master data in the core systems used by business users; it is where master data is created first; and it is often a common source of concern for many organizations since this is where poor (MDM) process integrity fails.

Analytical MDM centers on assuring “single view” of master data in the downstream data warehouse used most often to supply the data for a business intelligence (BI) solution.  We coined the term a few years ago to highlight the overlap (and differences) between the two environments even though “single view” seemed to be a common goal or requirement.

There are differences in what is called, “master data” in the two environments (hierarchy), and also a big difference in how “governance” is effected.  There are similarities in the use of some (not all) technology, most especially related to data quality and data transformation.

At our recent MDM Summit this topic was of great interest.  Hopefully we have provided “the” note that will close out most, if not all, of the open questions.  There were two other notes just published summarizing some common questions users shared regarding MDM.  Ted Friedman just published Q&A for Data Quality and Data Integration From Gartner’s 2009 Master Data Management Summit.  Don’t be mistaken – this is not all about technology; much of data quality is dependent on “context”; the reason why the data is used and the understanding the business user brings to the “question” has a huge impact on what the data means. 

And John Radcliffe published Q&A on Organizing for MDM From Gartner’s 2009 Master Data Management Summit in which he explores some of the issues related to the organizing for MDM.  Organizing for MDM is never too far from process, and governance, so these questions can get pretty complex, and very quickly. 

Want to meet and chat about all things MDM?  I will be at the Gartner Application Architecture, Development, & Integration Summit  in Las Vegas, NV, December 7^th-9^th. 

Wrigley’s Kristen Daihes, Senior Manager of Global Sourcing, presented how Wrigley had used IBM’s solution LogicNet Plus to help with strategic supply chain network design issues.  Such issues are often complex, periodically evaluated, and can result in capital investment activates (such as building, or closing, of plants and warehouses etc.)  Such programs are also very data intensive since complex models of the supply chain, and its possible behavior over time, has to be built.

Kristen described the use of this tool.  She highlighted how fully 60% of the process/project time was spent on “identifying, collecting, and validating data” that is used to model and create a view of the supply chain, while a further 20% was spent actually analyzing the output.  On first pass this might sound odd – why would users spend 3 times the amount of time on “data” and so little on the part that yields the greatest value to the business?

It turns out that the quality of the output (of the model) is highly dependent on the quality of the inputs – the data.  Remember that old adage, “garbage in, garbage out”?  So what kind of data are we talking about? 

• Product, relationships/structures, rules (used on quantity, popularity, affectivity etc.)
• Locations, and all the pertinent constraints (limits, boundaries, throughput limiters, costs etc)
• Lanes, or a representation of how products move from “a” to “b”, and all the pertinent constraints (alternative modes, costs, other constraints, availability etc.)
• Demand (actual orders short term and forecasted demand for many months, even years)
• Resource capacity (suppliers, plants, key bottleneck resources)
• Other data such as calendar, units of measure, currency, tax, duty draw back, etc.

Much of this data is master data: products, parts, and locations etc.  Much is reference data (acts like master data, but not a core entity, such as units of measure, currency conversions etc.  And there are other data that does not fall into either definition easily (such as calendar).  So the learning was this: an affective MDM and broader Enterprise Information Management strategy will help simplify the heavy data management side of such a complex supply chain activity.  

I found it interesting that I took a client inquiry last week that went like this: We see great value in mastering “single view” of master data with MDM – we would like to extend such governance efforts to other data types – how do we go about this?”  There are also many inquiries from users who are trying to establish active governance disciplines as they migrate their business data from legacy to ERP systems. 

This is a worthy question – that seeks to leverage the focus and success of MDM (its discipline) other data that also is shared across the business, but is not a core entity (master data).  This is also a good idea (where sought) but this implies a shift from (only) MDM to a broader EIM strategy.  Enterprise-wide metadata management would be used to manage metadata.  Content Management would be used to manage content.  Enterprise Information Management (EIM) should be used to link all these efforts together and ensure no duplication of effort.

So back to Wrigley presentation…

Another big idea I took away from this very effective presentation was that the quality of data, not just master data, is so critical to many initiatives that relay on data to be shared across enterprises or firewalls.  So much of SCM is like this; so little ERP (HCM, Finance) and CRM is quite like this.  CRM has evolved in this direction in the last few years, but SCM is streets ahead in this regard.  Only Procurement is as close to SCM – simply because procurement and SCM are twin brothers (perhaps CRM is a cousin).  Procurement and SCM have been sharing data with external trading partners for years, far in excess of simplistic transactional exchange.

Another big idea I heard clear from Wrigley’s Kristen was this: before you start implementing any aspect of SCM/SCP, be clear you know what the business question is that is being asked.  She explained that Wrigley had a clear idea of what the business really wanted to evaluate and test in terms of business changes.  Without such clarity, the initiative could have failed.  This is because you will not have guardrails enough to help you gather the right data, and build the right model.  The business won’t know what to do with the data – and won’t relate it to a question, an answer, or benefit.

It was a valuable day out of the office.

“…MDM projects harmonize after the fact, that they pull from the source system, they merge and match, they run  rules and check results on the back end.” 

This was in reference to a specific user implementation of MDM, apparently.  I see a lot of this type of activity, and I would not call this MDM.  I would call it data quality activity, or a data quality project: You do something “after the fact”, to clean the data up, as oppose to change the process that created the data in the first place. ‘Real’ MDM is more focussed on the process change AND data quality.  Data quality is a capability that has to be applied.

Analytical MDM is all about cleaning up the master data after event; post transaction; downstream, in a data warehouse in support of building reports etc.  Operational MDM is all about cleaning the data source (which COULD include, the point of authorship), and also about cleaning up the processes by which the poor quality data would have been created in the first place.  That is the main difference between the two efforts.  

Also, MDM is not a project.  Projects have an end or complete data; MDM does not.  MDM is a way of managing certain types of data – so it does not “end”, it just changes, and continues to change as the business changes.

Overall the article goes on to clarify that the description was an MDM “case study” and the article did represent MDM fairly, but I was not quite “on board” with the earlier part, as you can see above.  Seems like the author wanted to create a little hype in order to drive hits to his site.  Well, we are all subject to that little sin now and then…

Submit a customer case study to win the Gartner MDM Excellence Award 2009!  Go here: http://www.gartner.com/it/page.jsp?id=851612&tab=special&params=pg,zzz.html