Last week in the print edition of the Wall Street Journal (Sept 13, 2011) I spied an interesting article on the front page, entitled, “New Medical-Billing System Provides Precision: Nine Codes for Macaw Mishaps.”  Basically a new federally mandated set of codes are to be used nationally to provide for a classification system tracking all our bumps, scrapes, and other reasons for our visits to the doctor or hospital etc.  The current national set of codes was close to 19,000.  This set is to explode to 140,000.  Quite a jump.  I am sure such detail would provide interesting analysis for what ails us, in large groups over time, but the change in scale will create an administrative nightmare.  This updated set of standards has huge ramifications for governance of information, which we know is struggling to be sustained anyway.  Nothing like piling on more work as we go.  I wonder what the code for “too much stress at work” is?

Collectively I see a lot of folks coming to terms with defining, and re-defining, the role of “data governor”, “data steward”, and “data maintenance”.  I think this is a critical delineation of duties, yet so many organizations start with an “all in” definition and then struggle when it comes to execution.  I don’t think MDM, and any related information governance effort, can work unless these roles are clearly identified and resourced (sometimes by separate people).  I don’t mean to suggest that organizations need to build up their bureaucracy; it could be that these roles are staffed by one or two folks.  The point is that these roles exist today in every organization, but mostly informally and inconsistently.  These roles need to be explicit and the dependency of the roles needs to be made clear; I know what I am supposed to do in my home every evening; why not in our day to day work?

I spent another day this week with several other clients I was reminded, as I flew home, of two press pieces.  Jim Ericson over at Information Management, wrote a nice piece: MDM: No Roots, No Fruits.  The part of the article that resonated with this week’s most recent client interactions was this: “…[I]t’s easy to forget that most companies – by far — are only kicking the tires on MDM.”  This is just so true.  And this tells you just how far we have come with “data governance” – not very far.

This does not mean that there aren’t examples of other data that might be governed for specific purposes (think records management, digital media), but widely adopted, enterprise wide, primary information assets, broad based data governance does not exist.  And despite this many organizations are still “successful” in that they still ship the goods, still tax the citizen, or still power the washing machine.  Talk about a tough message to sell.

In support of Jim’s point I noted a survey Kalido published recently.  In The State of Data Governance 2011” they reported, among other things, “Despite nearly seven years extolling the virtues of managing master data by analysts and vendors, 49% of companies still do not manage master data anywhere but in their disparate applications.”  Well I have to disagree!  I think a figure of 49% is very flattering.  I think the vast majority of organizations that could take advantage of information governance programs like MDM are “not there yet”.  I would estimate nearer 80% of companies are not managing their master data properly.  A figure nearer 80% seems to jive better with the level of inquiries we saw a few years ago, the much higher level we see now, and the type of organizations we talk with.

On a more positive note Heiler also published a survey of organizations using MDM solutions to help master product data.  The survey is called “Return on Investment (ROI) of Product Information for Multichannel” and by definition, this is a segment of the overall MDM market.  The good news is that the survey showed some interesting data supporting the business case for how an MDM approach helps organizations achieve their business goals more so then without MDM.

“Conversion rates”, that is, the rate at which a prospect turns into a customer transaction, was reportedly higher for organizations adopting MDM of Product Data.  But oddly only manufacturers surveyed suggested this; retailers suggested a much small impact of governed data on conversion rates.  Then again, the maturity of MDM in retailers lags that of manufactures by quite a large margin (I surmise) so I wonder if the respondents really answered the question that was asked of them.  I would have to explore the responses (and organizations) more fully to get to the bottom of the difference.

Either way both survey’s expose some good information, and provide more food for thought.  I am still really excited at the amount of work going on, but there is so much more ahead of us than there is behind – for most of us.

I spent a few minutes comparing and contrasting MDM to BI and CRM and other business applications, programs, and disciplines.  We explored the purpose of MDM, and how it would provide a foundation for all manner of applications that consume data.  Toward the end of my “pitch” I mentioned how adopting MDM will likely lead to discovery of how business rules in the source applications will eventually be questioned, and that they may need to be “controlled” by the MDM program.  As I paused for oxygen, the client jumped in:

“Hang on, I don’t want to do all that stuff.  I think the solution is only a little, simple database in the center of the IT systems that maintains “single version of the truth”, that every user and system can interrogate for any purpose, that publishes out, as needed, good, clean data”.

OK, this is where my mind ran ahead of the words coming out of my mouth.  This sounds like a logical request, but how on earth would this work?  Surely, if this solution were this simple, why did MDM come about?  Could a full blown MDM implementation be serviced by “just” an uber table that stored links and maps between the erroneous data in the source systems and all new resulting systems, including the new CRM and BI tools, yet to be deployed in 2012 and 2013?  Of course, the answer is “technically yes, this could be done, but that is not the point”.  The reason why we are all in the pickle (of inconsistent data across heterogeneous landscape) is because of the number of one-off, “little projects” that were designed to solve this problem that became, themselves, silos that did not replace anything, any role, or any service.  Technology, my dear client, is the least important thing here.  Governance IS needed – there is no way around it.  But how could I bring us back to reality?

We back tracked – and talked about governance, stewardship, roles, responsibility, data ownership, process integrity, business outcomes, and so on.  As we talked more, I could hear the clients wooden/noisy chair gradually lean back on its springs, and could imagine the client coming out in a cold sweat.  I ruined her day.  She did not want to hear a Gartner analysts tell her that this was going to be hard work.  She wanted a simple IT answer to a business problem, make her decision simple.  I felt the wind leave her sails, my customer service score falling, as she put the hand set back on the phone cradle.

I hate calls like this.  On a few, rare occasions, the client spots the mismatch in expectations and they are re-set, and success is sought.  For too many, the short cut is the easy way out – and I get the call 2 years later after the initial “pickle’ as gone very rotten.  And there maybe even less money (and energy) left to tackle the real issue.

I am convinced that in five years time I won’t be researching MDM – but I will be researching information governance.  This is not a hope, or a desire, but recognition that finally, something as important as information governance will be legitimate, real, doable, at a cost that won’t be prohibitive, and in a manner that will be repeatable.  MDM is the catalyst.

Gartner has always said (the stated position) that MDM is a discipline, and one that includes or is focused on governance (of master data).  As such:

a)    No real MDM program exist without governance

b)    Governance of (just) master data is what MDM is

c)    You can’t do MDM without a hefty dose of governance

d)    You can govern a whole lot of information that is not master data (but that would be called something else, not MDM)

I have often said the following:  “Anybody asking to talk about information governance is really not saying much”.  The phrase has no context that is actionable.  What kind of information is to be governed?  As the context shifts across analytical data, transaction data, master data, metadata, content, records, “big data”, social data, so the driver for governance changes, the business case and sponsor too.  The technology needed to support it changes also.  I like to say that information governance is pretty much meaningless.

Most MDM offerings in the market today really only do a partial job at supporting stewardship (the operational side of governance) of the data for which they are responsible for.  No single vendor has a solution that is complete for governance all the mater data across the organization.  This gap – between the discipline of MDM that needs support for governance, and MDM tools that lack the depth, led to the emergence and rise of specific solutions targeted at governance.  This is Kalido’s Data Governance Director.  Other vendors have moved toward this space, but generally at an alarmingly slow pace.

However, trying to sell a solution for “information governance” is one thing.  Trying to sell the same tools to help establish governance of master data, as part of an MDM program, is something else.  Kalido is bundling their two offerings (governance and MDM) to deliver, basically, what is closer to what Gartner defines as Master Data Management.  This might be seen as a good thing – but for me it is recognition of the following:

• “Information governance” has to have a clear line of site to a business pain point, a business sponsor, a business case.
• MDM is one of the newest, hottest places where information governance tools can find applicability

In another orbit we have the whole world of enterprise business applications, such as ERP, SCM, CRM, Procurement, PLM, and various other industry packaged applications.  I include packaged applications, development applications and everything between.  In this orbit information governance is hardly talked about, yet all these application need their own application specific data, and they also need a copy of master data to work too.

Governing all this information for specific applications is another dimension entirely.  This is not what MDM solutions do- it is not what the MDM discipline does – it is what the business applications are supposed to do (we all thought) all along but generally didn’t (and still don’t) do well.  When will information governance come to those applications?  Soon, I hope.

I think “information governance” is coming – but not just yet.  I plan to be there when it happens.

I would go a little further however, and I might offer up: “ERP applications should be designed and implemented to support the needs of the Enterprise – and that basically means that the ERP of the last 15 or so years comes up short in this regard”.

I have blogged a number of times on the issue that “ERP” does not mean the same thing for every user (let along reader).  But, putting that aside for now, ERP is not (for the vast majority of users) an solution to provide business process or application support for the entire enterprise.  Despite the millions we have all spent on “ERP” it is any or all of the following:

a)    A big sucking sound that takes up 5 or more years of my life

b)    A bus ticket for consultants to get their children through college

c)    Sold as a single application tool for my business, but never was

d)    Just a pretty important, but not the only, business application with its own humongous data model (among other data models in my business).

In reality ERP was not a bad thing – it just get’s called many bad names because it is just such a fluid concept and as such, it gets all the flack.  If it really was one single application stack with one single data model, that did support every major business process that drives the organizations, it would not attract all the headlines.  Also we would be on the moon since there would not be any oxygen left due to the sucking sound.

Seriously, I think that ERP was really only ever a means to “single version of the financial transactions”.  And I think ERP has proven that it can do this – hands down.  It can also do other things, such as help with HCM, and indirect material procurement.  But when it comes to business processes spanning CRM, SCM, PLM, Procurement (direct materials/goods for resale) ERP started to shows its weaknesses and variability.  ERP became a meal, not just a single flavor.

And Don’s article nicely highlights how the complexities we get with ERP lead to complexities in business intelligence.  And things like MDM, itself a governance program for some information assets such as master data, will (over time) help to get the data cleaned up at source.

a)    Issues related to “what is master data” leading up to and including, what specific attributes are “in” and what attributes are “out”.  These issues intersect with the idea that MDM seeks to govern those attributes that are common and widely consumed across the organization, whereas business applications would remain in control of those “other attributes” that may describe an object but in context only of that application.  Users of ERP systems know this to their peril.

b)    Issues related to how the definition of “master data” (at the attribute level) changes over time as the business changes over time.  I talk with many user organizations about MDM, and for about 1 in 10, a forward thinking user will ask, “so what happens when an attribute that was NOT needed to define master data suddenly changes into an attribute that is needed?”  This is a particularly forward thinking question – since it understands that MDM is not a static project; it is an ongoing process.  The problem is, if this condition arises (and it does), how does MDM help?

It turns out that a) and b) above are not two steps that take place in serial, but they take place in an iterative cycle, over and over, on an ongoing basis.  Over time, the definition of what qualifies and so needs governance via MDM, will change. 

However, a best practice for MDM is that the data model describing master data needs to be “minimalist”.  This means that the data model is neither inclusive of all data in the organization, nor even the data specific to applications that are used to augment the core data for specific purposes.  But how then can an MDM program manage the shift from application specific governance of an attribute to a central MDM governance program?  That is where the essence of my comment came from.

Mature users have used metadata models to capture initially the core attributes that define master data.  Very soon after that metadata model also tracks and defines the next set of data that is very close, in terms of user and context, that is often managed at the application level.  As such, the metadata model in support of MDM will always tend to be larger and more expansive than just the master data model.

Over time (and this has been seen in more mature MDM programs), the metadata model gets larger and larger, in order to help the MDM program sustain itself.  But as this metadata model gets larger, eventually it starts to be re-used for other purposes.  In fact (and this is where I have yet to see real examples but I am sure they are coming), the emerging metadata model, founded on a core and successful MDM program, starts to support the active governance and re-use of other data related to master data.  As such, the foundation of an EIM program is enhanced.  MDM, which is a form of EIM but only focused on master data, becomes a clear stepping stone to a broader EIM program. 

Not all MDM vendors though are very good at exposing metadata about their master data models; and few of these MDM vendors are good at providing tools to manager metadata, let alone metadata from other applications.  So this evolution provides a fertile ground for partnering and development.

Hopefully this provides a better explanation of the comment I made.  If anyone has first hand examples you can share, it would help.  Alas all our inquiries (the hundreds we do every year) are confidential so I can’t share names.  Suffice to say I have seen this evolution taking place in more mature environments.

Hope to see you at next week’s Gartner MDM Summit!  We could argue some more on the links and intersections of MDM and metadata management.  Mark Beyer will be presenting on the topic.  See you there!

Well I am half way through the book, and I have an observation.  Firstly, I really like the book.  The content so far is excellent, applicable to MDM, and useful – it could help users understand what they need to do to set up, “data governance”.  The problem I have is the graphic that represents the unified process.  This issue also relates to “data governance” since, for me, this is a meaningless phrase.  It does not tell me, the reader, what kind of data we are talking about.  And if the intent is “all” data this is also meaningless since no organization will govern all their data.

The problem with the unified model is this: there are a number of steps that are “universal” in that, independent of what kind of data is to be governed, these steps are the same.  Then there are four choices – Master Data Governance, Govern Analytics, Manage Security and Privacy, and Manage Lifecycle of Information.  Here are my issues with this framework:

• Master Data Management is just one “to do” within Master Data Governance.  For Gartner, MDM is a discipline that spans the entire unified processes, starting with how MDM meets or supports business goals and objectives.  Perhaps IBM should re-name this box, “Implement Master Data Hubs” or something that relates to the technology, not the whole program.
• “Manage Lifecycle of Information” is actually written as “content management”.  Content is not the only type of data that has a “lifecycle”.  So the implication is that a key governance concept is relegated to “content” only.
• “Manage Analytics” is part of what should happen within a BI/data warehouse context (historically) and with a Performance Management context (forward looking), and specifically within a BI Competency Center.  So the application of the Governance Unified Process overlaps with this work.

What ought to be explained is that the “unified process” has different flavors.  MDM could be one implementation of the unified process.  If another type of data is to be governed, then the model is adapted: some of the steps are expanded (the common ones), and some would be specific (the optional ones).  I think the IBM model actually supports this (and I may find some text in the second half of the book that says this) but the graphic is mighty confusing.

How can you implement MDM if you don’t include?

• Define business problem
• Obtain Executive Sponsorship
• Conduct Maturity Assessment
• Build Roadmap
• Establish Organizational Blueprint etc.

I will say again – it is good material and very meaningful.  I just hope that readers don’t get confused with the positioning of some of the concepts.  Let’s see if I change my tune by the end of the book.

IBM InfoStreams sounds like a cool product – to embed real time analytics into streaming data certainly seems to provide a new level of awareness to users and timely decision making.  Fraud detection seems an obvious beneficiary to this technology. 

The “Big Data” theme was everywhere today.  Odd how much of that does not look new.  Other than Hadoop, which seems to represent processing capability, there is not much new here that we didn’t really talk about last year.  Maybe Big Data just formalizes a lot of bits and technologies that were disparate and disconnected last year…

On of the real-time survey’s IBM did through the event was very interesting.  Apparently 71% of those that responded suggested that business decisions were made on information that was “un-trusted”.  This does not mean wholly inaccurate data – but not totally verified or governed.  That is a startling response.  But is it so bad?  I guess not totally bad; but competitive victory could be assured of business decisions could be based on a little more trusted data – that was the point.

Steve made some great points about ‘embedding analytics into the business process’ which is perfectly in line with Gartner’s Performance Management research.  In fact, it would seem that Gartner’s BI, Analytics and Performance Management platform is very close to IBM’s platform for supporting BAO.  It’s a shame that “PM” didn’t quite make it; and worse, IBM is positioning PM as an evolutionary pre-curser to BAO.  I rather see BOA as an example of what we would have called “BI, Analytics and PM” but there you are.  That’s why I am not in marketing anymore.

The guest speaker was none other than Dr. Atul Gawande.  This was a really great keynote.  He is clearly an effective orator; the quieter he spoke, and more the large audience hustled down and listened even more intently.  He explores some of his findings in relation to how information helps the US armed forces in reducing fatalities from combat.  A very worthwhile keynote.

At the end of the day 2 there was an InfoSphere Keynote.  This sported several IBM customers presenting on “data integration and governance”, MDM, and business analytics.  Martin Wildberger did a great job highlighting the link between InfoSphere (spanning data integration, governance, MDM etc) supports the IBM vision for BAO.  It was a shame this message was left to a meager audience by comparison to the opening on Monday.  I thought this message could be been clearer sooner in the event.

The one wrinkle in the InfoSphere keynote I spotted came to me when Martin ended his pitch on a visionary thing.  He asked himself, how will IBM start to manage structured and unstructured data, which harked back to the question that drew such a complicated answer from IBM at the opening keynote.  His answer?  Big Data.  Big Data seems to be the preferred battle ground where this self declared battle will be fought; as wave after wave of unstructured data is brought to the fingertips of the business user in a form for them to build “ad hoc” meta-maps in order to relate an insight (from outside the organization) to information (inside the organization).  The problem is – few are thinking about how on earth such ad hoc meta-maps will be managed?  What happens if every user creates the same or virtually same meta-map?  What a right mess we will soon have.  Big Data could very well create Big Mess.  I would have much preferred to have seen an evolutionary road map that starts with solving the most basic of organizational issues – defining and standing up a “governance appliance”.  Well, there is always next year…

Stone Soup rules (I think!)

IBM IOD is always a fun event.  Over 10,000 folks are in Las Vegas (according to IBM) – including DBA’s, enterprise architects, master data managers, BI managers and so on.  The key-notes are always good to follow – to see what was included and to spot what was missed. 

Several years ago when “Enterprise Information Management” was the hot inflection point, I spotted the lack of “MDM” at the center of the Information Management strategy and messaging.  MDM was “there” but under the covers and not elevated to key note worthy status.  This year MDM was clearly present at the key note.  Arvind Krishna’s “information supply chain” clearly showed MDM and master data front and center:

The slide was repeated in the press conference that followed the main Monday key-note.

However, the IBM’ers used the terms “single version of the truth” and “mastering” in several new ways.  It seems that, “mastering information” and other things  is in vogue.  Perhaps this is another inflection point (joke)?

In fact I spotted something interesting.  Several speakers were talking of, “mastering analytics” and “single view of definitions like revenue” and I realized that the goal of MDM, that of delivering to the business a single version of the truth, is becoming rampant and is being applied to lots of different areas.  This is good news (MDM is proving to be a great idea, not just a good idea) but it can lead to confusion.  What discipline should end user organizations adopt to assure single version of these other things?  Does ERP yield single version of “revenue”?  Does BI?  Does EIM?   Does IBM’s Business Analytics and Optimization (BAO)?  Either way I give IBM good marks for this leverage.

On the negative side I saw a lost opportunity with respect to governance.  Governance clearly was not an “in vogue” this year, there was not governance “inflection point”, though I saw Andy Warzecha, walking about the place.  One of the video recorded questions that was posed during the key-note concerned how, and if, IBM will ever help users manage and/or govern structured and unstructured data.  I won’t ponder the meaning of “unstructured data” (you know what I mean) but this was a great question.  The intent was meant to draw out a response for a specific solution.  Predictably the lead for Information Management proposed a solution from the IM side; and the leader of the CM team offered a CM solution.  The leader of business analytics chipped in with some intersection with social media (an example of unstructured data) and analytics (structured).  What was missing as an opportunity to offer a single, unified solution.  If BOA is to be adopted as a strategy, silod solutions for governance won’t cut it.

IBM continues to talk about governance in its silos, not across silos.  I think that IBM is not exploiting an opportunity to reconcile the disparate governance efforts into a single, unified governance offering that would intersect and manage lots of different types of data.  IBM knows what is needed- lots of metadata management, that nice business glossary they have, lots of analytics, lots of workflow, lots of business rules, lots of data integration, and data quality.  But why does MDM no announce a unifying “governance platform” or application?  One might think that IBM can make as much money as it can by NOT solving this problem with one solution.  Perhaps it is better to manage by silos. Perhaps.  Not.

Another observation: at the Data Management key note Arvind did a great job beating his chest over the big and the bad Blue boxes.  There was not much hint of business value, just savings to IT and efficiencies.  I was hoping for more of a business led dialog.  It is a shame that Rob get’s to talk about business value (Cognos) and Arvind gets stuck in the weeds.  Maybe in the InfoSphere key note tomorrow there will be more business value links.    If not, maybe I can offer myself for the key note ha ha. 

• Your enterprise architects should be business oriented, not technologies
• Governance is key (developers do not like to re-use stuff, they want to create new stuff all the time)
• Standards are useful  but are not a silver bullet
•  Think process, process, and process, then think process again

The point being that architecture has to think big (eventually) if the right services are to be designed for reuse across the landscape.  So “end to end” process view was important, and necessary, for process integrity to be baked in. 

I would have asked Brigadier Clacher a question – but for this summary slide.  He finally said those magic words, “all this stuff needed some key enabling elements, such as MDM”.  His pitch was good, but it focused on process and BPM; there was no real emphasis on data and MDM.  The entire pitch probably could have been given from a data perspective, with less emphasis on process, but that would have been a different pitch. 

Its just odd that SOA folks talk a good game with respect to BPM and process, yet they just don’t give data the time of day.  Let’s see how my “MDM and SOA” pitch goes later today