I did a vendor briefing with a vendor today. The topic was “how we support end user needs for managing metadata”. This vendor would be found if you were looking for a metadata repository.
As we went through the briefing, the vendor was presenting the range of capability supported by their tools and technology. I asked two key (for me) questions:
- Where in the organization have you (the vendor) focused your efforts – or, put another way, what specific kind of business problems have you solved most frequently?
- What do you (the vendor) do to support information governance?
The answers to both were, for me, illuminating. And perhaps at the risk of exposing my ignorance of the space, I wanted to share my thoughts on the not uncommon responses from this and vendors in this space.
It’s in the Data Warehouse
Despite nicely describing the role and value to the business of better managed data in general, and metadata specifically, the vendor’s experience was predominantly in helping end user organizations set up and understand the data that goes into a data warehouse, in support of Business Intelligence (BI) and analytics.
This is valuable for sure. But it is also predictable; very IT focused, and despite the terminology used, not necessarily the place where the greatest value from metadata and its management should or could be realized. The bottom line is the use of such tools is a “no brainer” as it pertains to gathering up data from lots of different (i.e. heterogeneous) systems to load up a data warehouse to enable reporting, and to meet compliance and regulatory requirements. But this is not the same as tackling the problems in the source data in the core business systems. The former tends to cope with the disease (from a lack of information governance in operational business systems) and the latter seeks to tackle the problem head on.
Additionally, this vendor (like many others in the same space), suggest that their tools are business friendly. But in my experience, business users have little interest in the additional work, unrelated to what they do every day, to clean up date from erroneous ETL scripts and metadata models that operate in a data warehouse environment. “Isn’t that what IT should be doing?” The data warehouse is so far removed from their reality that this work always seems to be onerous and of little value (to the business users).
Vendor (and users, mostly IT) often report how hard it is to keep business users involved. No kidding – I would not want to spend my time doing this stuff. The threat of compliance tends to focus the mind, however. But the focus is very different from the idea of using metadata and its management to help the organization drive revenue and/or improve customer service – which it would be if metadata management were to help with governing data in business systems.
It’s the business data that counts, not the metadata
In their overview the vendor included a dialog for how its tools help its customers with the stewardship (policy enforcement) and governance (policy setting) of metadata. I had to check that – the governance of metadata? Of course I know metadata needs to be governed. But I asked: if the focus is in sustaining consistency in the actual business data, it’s the business data and various competing authority models in core business systems that need to be governed. This was emphatically not the focus. The vendor reinforced that they govern the metadata – not the business data.
In support of the vendor’s experience, their response is reasonable, but still missing the point (for me). In any DW implementation there is a lot of technical metadata, and there is some business metadata (think ‘customer’ definition). But how many business users spend their working hours working on metadata definition on a data warehouse? They struggle to govern properly the actual data (and its business metadata) in the core business systems.
This for me is a major opportunity. Metadata management tools have, for a lot of users, been focused on very specific areas, and few of them are really focused on the governance of operational business data. Its often down in the warehouse, or in archived data for e-Discovery, or perhaps for glossaries shared by content oriented data models. All are good and can add value. All I am doing is calling out a difference in focus and opportunity. And I don’t mean only describing and defining the metadata for an application integration effort. I mean, using metadata and its management to help sustain the policies that enforce compliance for how business data (not metadata) is created in the first instance, in core business systems. Right first time – rather than cleaning the rubbish up, over and over.
I look forward to the day that metadata management tools (and vendors) get into bed with operational business systems and applications, and help make (operational) information governance work, to help improve business outcomes.