Operational MDM centers on assuring “single view” of master data in the core systems used by business users; it is where master data is created first; and it is often a common source of concern for many organizations since this is where poor (MDM) process integrity fails.

Analytical MDM centers on assuring “single view” of master data in the downstream data warehouse used most often to supply the data for a business intelligence (BI) solution.  We coined the term a few years ago to highlight the overlap (and differences) between the two environments even though “single view” seemed to be a common goal or requirement.

There are differences in what is called, “master data” in the two environments (hierarchy), and also a big difference in how “governance” is effected.  There are similarities in the use of some (not all) technology, most especially related to data quality and data transformation.

At our recent MDM Summit this topic was of great interest.  Hopefully we have provided “the” note that will close out most, if not all, of the open questions.  There were two other notes just published summarizing some common questions users shared regarding MDM.  Ted Friedman just published Q&A for Data Quality and Data Integration From Gartner’s 2009 Master Data Management Summit.  Don’t be mistaken – this is not all about technology; much of data quality is dependent on “context”; the reason why the data is used and the understanding the business user brings to the “question” has a huge impact on what the data means. 

And John Radcliffe published Q&A on Organizing for MDM From Gartner’s 2009 Master Data Management Summit in which he explores some of the issues related to the organizing for MDM.  Organizing for MDM is never too far from process, and governance, so these questions can get pretty complex, and very quickly. 

Want to meet and chat about all things MDM?  I will be at the Gartner Application Architecture, Development, & Integration Summit  in Las Vegas, NV, December 7^th-9^th. 

In analytical MDM there is no message or alert sent to a business user, that is resolved during the normal cycle of work events.  It would be more of a project, rather than a process.  This is a major difference between the two domains; and a difference that must be taken heed of.  Too many users miss the point and assume that what works with BI will work with MDM.  I hear, too often, the comment, “I can ‘do’ MDM with my BI/data warehouse”.  This is partially true in terms of the mechanics; but in terms of the process.

Mark is very fair, and logical, and he says that there is no governance in BI.  He is very right, since for him, governance implies activity (not just mechanical rules).  So we do agree; we just differ how we name the condition.

Guided Governance versus Mechanical Governance

Given that analytical MDM and operational MDM share common mechanics, it is easy for BI technology to be used (i.e . demonstrated) to support governance.  The problem is that this is not the entire solution.  I saw a vendor demonstrate a new “governance application” that showed, reasonably nicely, numerous mechanical activities a user would follow day-to-day.  These mechanics supports an operational MDM environment – they could be seen to support day-to-day, business user activities that they would have executed anyway.

What was missing from the briefing was, as I said on the briefing, “guided governance”.  The vendor had no demonstrable technology that provided the business user with analytics and metrics related to data quality, process exaction, or process design.  As such, there were no guard rails or searchlights highlighting where business users (i.e. stewards) need to work their magic.  It is like piloting an aircraft with a manual; everything goes great until environmental conditions change for which the manual cannot predict and instrumentation is needed to guide the user to make a decision.  As with the example, bad news is what happens as a result of this lack of instrumentation.

I called this instrumented environment, “guided governance” versus the more common “mechanical governance” that this, and most other vendors, is initially focused on.  It won’t be long before users help the vendors out and tell them what they need (once they start flying those planes).

• identify erroneous master data that need resolution (in order to yield “single view”)
• explicit intent to clean up master data at source
• resulting output (clean master data) is used in any and all business applications used by line of business

The second item could be said another way: explicit intent to clean up the business processes that crate the bad data in the first place.

As such, operational MDM involves business users in real work (over time, less then what had been executed before MDM) to resolve issues with bad master data.  Process chances result in possible changes to application strategy; changes have to be made to legacy and packaged applications that hitherto fore had not awareness of the MDM discipline.

A slightly weaker pattern related to MDM “use case” was visible in Business Intelligence (BI) land.  When IT builds a data warehouse to support BI it has to identify data sources, and that data is gathered together (federated), and cleaned up (merged, transformed) and stored in the data warehouse – on which all manner of analysis is performed.  Mechanically, some of the cleaning routines and rules generated from them, look and smell like the same as those that would be used in operational MDM.  But, there were also some differences, and this is what helped us create the name for this next pattern (analytical MDM).  The common characteristics that describe this use case were (and still are):

• identify erroneous master data (among all the other data targeted for the data warehouse) that need resolution (in order to yield, in the data warehouse, “single view”)
• no desire to clean up master data at source
• resulting output (clean master data, and all the other data to be stored in the data warehouse), are not used in any business application, only in the BI applications using the data warehouse.

What gets real interesting with analytical MDM is the third bullet.  In the last few years more and more applications have been built on this “BI data warehouse” that create new data; and this data has to be stored, managed, and governed, like any other business data.  As such, this “BI application” behaves like a business application.  This is important.  We need to keep things simple, and re-define things.  A BI application is one that is used for reporting and analysis only and does not create any new business data; a business application may also have analytics, but it does tend to create business data.

For BI applications, analytical MDM is adopted.  For BA, operational MDM is adopted.

There are at least two other use cases for MDM – but they are much weaker to discern.  I will follow up on those in another post.

See you next week at Gartner’s MDM Summit!

Over the last 5 or 6 years this focus has broadened in scope and has since spawned other drill down topics, such s Procurement MDM which has, itself, split into MDM of Supplier data and MDM of Purchased Part.  Holistically though, MDM of Product Data generally means mastery of “thing”.  Thing is a kind of ‘province’ in that most “things” have common characteristics or challenges; and characteristics and challenges across ‘provinces’ are not similar (think customer/supplier master data).

Well this forward looking user was just starting to roll out an MDM application (single solution, single vendor) for both buy-side product data as well as sell-side product data.  I say this is forward looking because to date most users, if they have gone this far, have assumed that a different solution and possibly different vendor, are needed for each.  Sell-side MDM is much more mature – the mastery of the data was separated from the business applications that created – and consumed the data – 6 or more years ago.  For buy-side, the procurement and spend data management vendors have not yet “let go” the data and so users on the buy side assume that procurement vendors and spend data management vendors are also providers of MDM of purchased part solutions.  Like sell side, the separation of the data and the application will take place – it is just taking a long time.

So this client has taken a leap of faith – they hope several things:

• The MDM vendor can in fact cop with the complexity of requirements for both buy and sell side
• The business applications and analytics solutions, used by business users, to guide selling and buying decisions, will easily integrated with emergent MDM infrastructure
• IT can create a new discipline that, when needed to develop and deploy new business applications and analytics, they do NOT develop or deploy them in the MDM application itself, but as part of the business application and analytics layer.

When users ask for more analytics and business process support the natural and obvious choice will be to “add it to the MDM application”.  After all (the logic will run), that is where the really good clean data resides.  But in adding business application and analytic logic to the MDM infrastructure layer, IT would be reverting back to what it has been doing for the last 20 years – that created the mess in the first place that led to the conclusion that some change in “Integration” needs to take place!

There is a trade-off that emerges: MDM leads to the separation of master data from business application/intelligence.  IT is shifting from 20 years of vertical silos (application plus data) to a new framework of horizontal services (application, analysis, information management).   In the process, and it is a long process to get from a) to b), the IT stack does get more complicated.  So short term costs could increase, flexibility may not accrue immediately, and complexity may increase also.  The pressure to get MDM firmly established for all important master data objects needs to be kept up else the business will not see the benefits and the risk is that MDM will fall from favor for a traditional (silod) business-as-usual approach.

So it will be an interesting time for this user: Great vision, great strategy, hopefully not too far ahead of where the vendors can meet the user needs.  I will be watching very closely – as will many of you, I know.

I have written about “rules” in an MDM context before.  How an organization masters rules is not a new topic; analysts have written about (business) rules engines before.  But MDM is bringing a focus to the topic again, in the context of:

• What type of rules are there?
• Who authors such rules?
• How are such rules used?
• If rules or parts of rules (or rule making processes) are re-used, how can this be made effective?

It is the last question that is making users of MDM conclude that rules and MDM need to be related more closely.  And because BI is immersed in rules, the connection between BI and MDM is more important.  A client inquiry brought this connection unto clear focus today with the following description:

“We have been developing BI for many years, and we recognize that rules that adjust data in some fashion exist at many places in our systems, from initial identification, transformation, load, then aggregation, and even ‘post BI’ since users then take data and drop it into a spreadsheet and do more manipulations.”

These rules, in the context of BI, are what constitute aspects of what I call, “passive governance”.  My college, Mark Beyer, would chuckle and correct me with, “Andrew, this is not governance at all” and Mark is correct; I am just a little kinder and I accept that these rules have a job, that is to clean up and process data in a way that should be meaningful to the user/business.  As such, when “active governance” is established (aka MDM) then rules are again created to clean up and process data in a way that is meaningful to the user/business.  The different is that in BI land, these rules are created and managed by IT independent of the operational system; with MDM these rules are defined by business and directly interacting with the operational system.

So the challenge for me is much clearer – after this one dialog with a client – spelled out the obvious.  Over time more and more users will rationalize that they need to apply MDM disciplines to rules that process master data; and also, beyond that, integrity and re-use of those rules, even those that process non master data, can also yield business benefit through more effective management.  The connection between mastery of business rules, and how MDM tools help, will only increase in strength in the next few years.

It seems the Rime of the Ancient Apps Manager/Mariner should be something like:

Rules, rules, every where,

And all the re-use did shrink,

Rules, rules, every where,

And not an object to link.

“…MDM projects harmonize after the fact, that they pull from the source system, they merge and match, they run  rules and check results on the back end.” 

This was in reference to a specific user implementation of MDM, apparently.  I see a lot of this type of activity, and I would not call this MDM.  I would call it data quality activity, or a data quality project: You do something “after the fact”, to clean the data up, as oppose to change the process that created the data in the first place. ‘Real’ MDM is more focussed on the process change AND data quality.  Data quality is a capability that has to be applied.

Analytical MDM is all about cleaning up the master data after event; post transaction; downstream, in a data warehouse in support of building reports etc.  Operational MDM is all about cleaning the data source (which COULD include, the point of authorship), and also about cleaning up the processes by which the poor quality data would have been created in the first place.  That is the main difference between the two efforts.  

Also, MDM is not a project.  Projects have an end or complete data; MDM does not.  MDM is a way of managing certain types of data – so it does not “end”, it just changes, and continues to change as the business changes.

Overall the article goes on to clarify that the description was an MDM “case study” and the article did represent MDM fairly, but I was not quite “on board” with the earlier part, as you can see above.  Seems like the author wanted to create a little hype in order to drive hits to his site.  Well, we are all subject to that little sin now and then…

Submit a customer case study to win the Gartner MDM Excellence Award 2009!  Go here: http://www.gartner.com/it/page.jsp?id=851612&tab=special&params=pg,zzz.html

Andy Hayler, CEO, The Information Difference, wrote this article, “A Bulldog Puppy Emerges”, in which he describes how Microsoft has adapted its strategy to meet the needs, your needs, for MDM.  As of now, the MDM solution will emerge as part SQL Server, a database offering.  Some of the points in the article that caught my attention, and worthy of review, are:

• allowing third parties to build add-ons for it, for example specific industry offerings, and this is something that Microsoft will encourage.
• As and when MDS does appear, it will be a significant entrant into the market based on the sheer market presence of Microsoft.
• it will have had a lot more customer exposure than most new MDM products.
• MDS will not, in its early stages, have the kind of full functionality and proven performance of some alternatives, its presence in the market alone is likely to apply pricing pressure to other vendors

Well, my MDM travels lead me to disagree in some fashion.  MDM is certainly applied to industy specific level needs, which come from differences found in data model, business processes supported, business models, use cases, domains, use job titles, buying preferences, and references.  As such industry specialization has not (yet) become a horizontal layer that can be “added” by API’s.  If it had, other vendors would have tried.  In fact some say they are, and they are not doing all that well.  As such, this is a really big idea but one that has a lot of challenges ahead of it.  It would be innovative, and disruptive, but every time we look at MDM it looks more complex, not simpler.  

The second item is very possible, given Microsoft’s size and scale.  So this will be a “spoiler” entrant, but will it really represent a disruptive force?  I tend to think not.  The public side of this vision just does not have enough flesh on the bone…

The third item reads true – and the article does report that 200 or more customers are working on the product.  But is this “a new MDM product”.  My feeling is that this will be an analytical MDM product if anything.  So yes, it might have more customers for that.  But this will not be an operational MDM offering automatically.  In fact, it is very possible that the first 10 customers that try to use this for operational MDM will highlight the gaps.  No analytical MDM engine has successfully (yet) moved over to the operational world – in anything other than a few implementations.

The last items follows on from the third.  MDM is not a database level conversation.  One of my interactions with a client week highlighted this.  A client, an IT leader, said to me (and I almost don’t paraphrase), “We want to highlight to the business how MDM is not focused on our ESB or database, but focused on the business”.  It is possible that Microsoft will bring some price pressure on the MDM market, but it remains to be seen.

I did say, “yet” several times in the above review of the article.  I might be proven wrong.  But to be proven wrong, there has to be some other elements in the strategy that the article, and Microsoft, have up their sleeve….  Governance is the critical part of MDM and perhaps partners will “add” this later.  As such, the part that Microsoft brings will be very low level and “tactical”.  Hardly grounds for disruptive market entrant….

Overall the event was really good.  There were a lot of end users focused on BI and how to get more value from the technology for the business.  Almost all client interactions were focused on this.  Clearly BI has broad appeal, but that link between business perceived realized value and investment remain remains a challenge.  The other Gartner analysts were rushed off their feet – as they tend to be at Gartner summits.  These smaller events, compared to our Symposia, are always more focused on specific topic and so the depth and quality of interaction is always greater – and all the more tiring for it though also more exhilarating (for me at least).

From an MDM perspective, this was a strange event.  I missed the 2008 Gartner BI Summit but had presented on MDM and Performance Management at the 2007 Gartner BI Summit.  Back then MDM was very hot and a lot of attendees wanted to know what MDM was, what it meant to BI and themselves, and how MDM would change or improve or alter BI; for sure, there was not a lot of active MDM work.  I would have assumed that this year, in 2009, with MDM being more widely recognized, that more BI oriented attendees would be “into MDM” more.  However, the majority of clients I spoke with that the event were a) aware of MDM, but b) not active currently, and (more importantly) c), trying to figure out when and where to get started.  So I would assume this means that MDM is more widespread as a topic now, and so a greater audience is looking at it.  But as the prospective audience gets larger, so it becomes clearer that there are many more users who need to “get on board”.  With the very entry level or basic dialog I saw in the user community, it is very important we recognize that though leaders and pioneers might be far ahead in their understanding and use of MDM, the majority of users are not there yet – by a long way.

Off home now – to try to get 3 days of your life back…

We have been using the term best-of-breed, specialist, or niche MDM vendor to denote a vendor that has focused on one specific master data domain.  The two most common data domains referred to by users is customer, and product.  One side bar that always get’s my goat: does a customer focused enterprise always start with MDM of Customer data?  The answer is a firm, “no!”  A customer focused firm that sells product and/or services may have reasons to start with MDM of Product data.  After all, if you don’t know exactly what it is you have to sell, how can you assure good customer service?  But I digress…

Best-of-breed MDM has not necessarily implied single industry focus.  It so happens that a focus on a specific domain generally leads to a focus on one or a small number of industries.  It seems that the greatest gap in terms of IP exists between customer and product domains, rather than any two industries.  However, best-of-breed MDM does mean “depth” in terms of functional capability to meet the needs of that domain/industry.  And given the number of end users I and my colleagues have spoken with so far, there is a huge, wide discrepancy in the complexity of issues customer face with MDM.  

Another characterization of best of breed MDM is that the vendor focuses on what we call operational MDM.  Operational MDM is form of MDM that explicitly seeks, as part of its governance routines, to clean up source data once errors are found in the business. 

Analytical MDM represents a form of MDM that exists in a BI context, to support cleansing of data in a data warehouse environment, where there is no explicit desire to clean up the source data.  I came up with the idea of analytical MDM to highlight the overlapping in technology and technique (use of data quality, cleansing, normalization etc) and differences (desire to clean source; not to clean source) across a continuum of use cases.  Though not accepted early on it seems that the term, and differences, is now widely accepted.

So what is a generalist MDM vendor?  There are in fact different types of generalists.  One might focus on analytical MDM only.  This means that the technology is used to model all kinds of master data (product, customer, person, hierarchy, and dimension data) but in a BI environment only.  Another might be a vendor that focuses on operational MDM but for many different types of data domain, though the functional capability for any one domain is not that “deep”.  

Users have to decide what they need to focus on – and this may change over time.  Business drivers may lead to the recognition that “deep MDM” skills are needed first hand to get to grips with very complex product data workflows, but later, a more general approach is needed to master other domains.  

So the need will change over time; and the vendors capability will evolve also.  As such, there is no “right” position to take.  Both, and all the variations in between, have a role to play in meeting user requirements.  The key perhaps is to align the right vendor approach to the need.

Of course, putting my vendor hat on, I have to answer, “But which vendor will win?”  It depends – on what we mean by “winning”!