From my colleague Mark Beyer, who speculates about how leadership in moving toward the logical data warehouse (LDW) will be received:
The logical data warehouse is already creating a stir in the traditional data warehouse market space. Less than 5% of clients with implemented warehouses that we speak with are pursuing three or more of the six aspects of a logical warehouse:
- data virtualization
- distributed processes
- active auditing and optimization
- service level negotiation
- ontological and taxonomic metadata
That means we are in a very early stage regarding the adoption trend, and vendors who are aggressively moving toward it are ahead of their customers. We have spoken with at least one end-user organization that has implemented the entire solution from scratch in their own best of breed approach. Life teaches us that one aspect of humility is to recognize that if a good idea is really good, more than one person has thought of it. With a far greater number of organizations pursuing less than a complete LDW, but multiple parts, it’s only a matter of time. You can count on a trend for adoption of the LDW coming to fruition; timing is the only thing in question at this point.
[ Added 3/22/12 – MA
For those not familiar with the LDW model, here is a working definition taken from
Does the 21st-Century “Big Data” Warehouse Mean the End of the Enterprise Data Warehouse? (G00213081)
This new type of warehouse — the LDW — is an information management and access engine that takes an architectural approach which de-emphasizes repositories in favor of new guidelines:
- The LDW follows a semantic directive to orchestrate the consolidation and sharing of information assets, as opposed to one that focuses exclusively on storing integrated datasets.
- The semantics are described by governance rules from data creation and use case business processes in a data management layer, instead of via a negotiated, static transformation process located within individual tools or platforms.
- Integration leverages both steady-state data assets in repositories and services in a flexible, audited model via the best available optimization and comprehension solution available.
Hope this helps clarify for those not familiar with the research note above.]
Gartner clients with currently successful warehouses already have plans to retrofit and adapt the infrastructure as demand arises. Some of these are a bit long in the tooth with seven years or more of success, while others are as little as eighteen months old. A major portion of the data warehouse population will resist shifting to the LDW simply because it contradicts the traditional notion of IT success. IT projects are supposed to have beginnings and endings, but data warehouses have never had endings—in other words, the data warehouse is supposed to constantly evolve and resistance to the LDW is almost guaranteed retirement of your existing warehouse.
So, when Gartner comments that pursuing the LDW is ahead of the rest of the market, that’s a caution that it’s important to create ample justification, education and training to present the case in a comprehensible and compelling fashion and not a caution against the strategy. But that is not a negative comment – quite the contrary. It’s a recognition of a tough but worthwhile struggle ahead—so it’s truly a caution to justify your pursuit of something difficult. For vendors and user clients alike, it’s a signal that leadership does not come without its challenges.
Think of it this way. When the party is already started and everyone else is invited but you, you cannot just show up late and uninvited to declare the party is beginning because “now” you have arrived. If you decided to wait for the party to start then come late, you sure know that you better bring some snacks. But you better hope some of the guests are still hungry. Leading a market is a dangerous and risky business—and it attracts customers just like you.
Gartner clients: for more detail on the LDW, see Century