Mike BlecharI recently received a response to my blog entitled “Down With the Uber-Repository, Long Live Metadata!” criticizing my recommendation to not try to consolidate all your metadata into one “uber-repository”. Interestingly, while the respondent mentioned some success his company has had with that approach, he then went on to lament that the amount of metadata being managed in the repository was incomplete and that management did not appear to see the value in expanding its use to all the other metadata in the enterprise. And as a pragmatist, my response would be – good for them! Different business opportunities and threats ought to factor heavily into the decision of the how much rigor and breadth of scope you should apply to your data (and metadata) management.
I tend to find that there are two main issues which are incorrectly getting bundled together when Gartner clients want to address metadata management. And while they seem to want to make this mostly a technological decision, the fact is that it should be secondary to other issues. The first and more fundamental question needs to be “which metadata needs to be managed, and with what degree of rigor”. Metadata is pervasive in the organization – so managing all metadata is not a viable option. Like all other forms of data, there will be a subset of the metadata which is more critical to the organization. The most obvious example is the metadata about the subset of your information assets involving your data, processes, architectures, etc, which you consider to be “master data”.
In other words, as part of your enterprise information management strategy, you need to identify the master metadata and provide a more rigorous management strategy for it than other forms of metadata. And like all other forms of master data, there needs to be a business case in terms of governance, risk, compliancy, cost and opportunity for spending the extra resources to manage that master metadata (see "Mastering Master Data Management*").
Typically, an organization places top priority on improving the usability of its most valuable information assets (for example, customer, product and supplier data). Generally speaking, the more valuable the information asset, the more metadata there will be about it, and the more valuable the metadata will be. In other words, metadata is what unlocks the value of data and, therefore, requires management attention. The most valuable information assets ought to receive more management attention than those which are less valuable.
This is a related but different issue to the one concerning how to manage the metadata. As indicated in Gartner research note “Applying Data Mart and Warehousing Concepts to Metadata Management*” there are many approaches to managing metadata ranging from leaving it uncoordinated in multiple places and tools, to consolidating subsets of it in metadata marts/warehouses for reporting purposes, to everything in between including bridges between metadata stores and real-time federation of metadata across stores. The key is applying the right approach to the right problem. While a single uber-repository for all metadata is not feasible, there will generally be a subset of metadata which needs to be managed in a robust manner – such as through physical consolidation or federation of master metadata for change impact (and other forms of) reporting.
Therefore, the take-away point here is that you ought to look at your master data management (and critical business process management and service-oriented architecture) efforts to identify which data is most important and make sure the ”master metadata” related to that information gets adequate attention – perhaps in terms of it being placed in a metadata mart/warehouse. Other metadata may best be managed using other approaches like bridging, federation, or simply looking in multiple places for metadata related to information assets. In the end, good metadata management involves evaluating risk, cost, opportunity and governance issues to make the right business decision on which metadata to spend the most resources managing, while also picking the most appropriate approach to meet those management needs – one size metadata management does not fit all!
*Available to Gartner clients or for a fee
Category: Uncategorized Tags:
4 responses so far ↓
1 uberVU - social comments November 1, 2009 at 3:48 pm
Social comments and analytics for this post…
This post was mentioned on Twitter by Barbra5510: Master Metadata Management: Different business opportunities and threats ought to factor heavily into the decis.. http://bit.ly/1YrsLU...
2 Luciano Quadraccia November 25, 2009 at 12:02 pm
re : The first and more fundamental question needs to be “which metadata needs to be managed, and with what degree of rigor”.
yes, agree, but the answer is that all metadata that is needed to build a system must be recorded and managed. Because that metadata represents the specifications of the system.
All such metadata is best placed into the same tool – so that the relationships can be made explicit in formal data structures.
If developers now build the system based on the metadata then there is no question about whether the metadata is right. The metadata reflects the system. Particularly if tools are used to automate construction of (parts of) the system.
Why on earth would one choose to scatter such metadata? The only reason I can think of is where the repositories are not capable.
So, instead we should be encouraging vendors to build capable repositories.
And, because such metadata must be created during the application development phase, it must be subjected to the same version control and configuration management processes as source code. Generally, metadata repositories have much weaker version control facilities than regular Version Control Systems, eg they are weak in branching and merging, whereas Version Control Systems usually only deal with files, ie cannot deal with metadata.
3 Luciano Quadraccia November 25, 2009 at 4:44 pm
Oh, one more point – let me reiterate from my first comment:
“as our systems started expanding outside the mainframe they did not continue to use the “uber repository” for their metadata. As a result these systems have a variety of problems that do not exist at all with the mainframe systems”
So, the new systems that do not use the uber repository have certain problems, the old-style systems that do use the repository do not, and you say “good for them” ???
4 Mike Blechar November 26, 2009 at 12:54 pm
We are not in disagreement about the value of metadata management and metadata repositories. However, where we differ is in the scope of use. Historically, two out of every three organizations who attempted to do the scope of metadata management you advise ended up killing the metadata management efforts in the first two years. Of the other third, I would argue that (like your organizations) 98% use their main enterprise repository for only subsets of the total metadata due to the issues involved in broad “all-inclusive” use scenarios.
I’ve written alot of research in the past 16 years at Gartner based on my personal experience in implementing and supporting “uber repositories” at 5 Fortune 1000 companies with management support ranging from limited to wholly committed. But in the end, it comes back to balancing return on investment and risk with cost. While different organizations may draw the line in slightly different places, in the end metadata management (like all other disciplines, including data management) cannot accomodate all enterprise assets. Therefore, it is necessary to decide what is “master metadata” and which is not, and how inter-related the management and reporting of the master metadata ought to be.
For example, I cannot comment on whether your organization made a good or bad decision to have non-mainframe systems be “oustide of the metadata managed in the repository”. But, my expectation would be that some of the mainframe – and some of the distributed – metadata ought to be documented in an integrated way in an enterprise repository and some not. I cannot comment on whether the cost of the problems caused by not documenting the non-mainframe applications you mentioned exceeds the cost of maintaining better information in the repository to cut some of those costs. This is where those responsible for metadata management need to provide the business case to management – but again, a statement that “all mainframe-related metadata” belongs in the repository (to exclude the distributed stuff for now) is not a statement I would support. A more intelligent look at the costs and risks involved for different mainframe metadata ought to be made to come up with a proper metadata management plan. That plan would need to look at whether some of the metadata (and which types) ought to be in a single integrated enterprise repository and which in other more focused repositories or left in the metadata stores of the technologies in which they are defined.
So, my “good for them” statement has more to do with making the most appropriate business decision…..We make these decisions every day of our personal lives in terms of the cost and types d closthes we wear and when to repair our homes and cars and to what degree we spend money on them. So, it should be no surprise metadata management (like all other business functions) ought to not be based on “perfection at a premium cost”, but managed by making sound, responsible business decisions on where to spend the time, effort and cost on metadata management in a selective way.
Leave a Comment