Michael Blechar

In my last post,  I mentioned a new research note I had just published (“Decision Framework for Evaluating Metadata Repositories*”). In that post I discussed a proven, successful approach for evaluating metadata repositories based on specific scopes of emphasis and levels of organizational maturity in terms of governance and metadata management practices. I have just released a companion set of research “Toolkit: Sample RFI and Vendor Rating Spreadsheet for Evaluating Metadata Repositories*” which includes a spreadsheet for rating and ranking repositories using the decision framework, as well as a sample set of RFI questions for gathering information to rate the solutions.

While I obviously cannot go into detail on the RFI here, I’d like to highlight three important things which repository selection teams should address before sending out the RFI to the vendors.These will streamline the selection process, prevent law suits from vendors who feel they were treated unfairly, and circumvent later debates and arguments amongst team members about the the rating criteria being used or which vendors should qualify as “finalists”.

Must Haves: Information managers will need to identify "must have" criteria which the vendor solution must support to be considered for selection. For example, a constraint may be that "metadata must be stored and managed in a relational database management system". These are useful for quickly eliminating vendors from consideration whose solutions do not meet these minimum requirements (and allowing them to understand why). However, organizations need to be careful when coming up with their list of "must haves," since the highest rated — or best compromise — solution may get eliminated based on what truly may not be must haves. In other words these need to only be “unconditional deal breakers”.

Level of Criteria Detail: The scoring spreadsheet does not need to go down to the most detailed level of selection criteria. The impact percentage of each low-level detail in the decision framework (at the "leaf level of the hierarchical rating tree") should be large enough to have an impact on the overall rating/ranking of vendors as opposed to having time-consuming evaluation of too-small criteria which have little effect on the total decision. In other words, given that the "must haves" will contain details which will eliminate the contenders who are misfits, there is generally no need to evaluate the differences in scoring for vendors on a criterion which only has a 1% or less total impact on the decision.

Minimum Rating: A suggested guideline is to establish before sending out the RFI a "minimum acceptable rating level." That might, for example, be 80% — meaning that if the vendor/repository score is anything less than this percent that solution will be considered as unsuitable for use. This allows the removal of vendors whose score falls short of "final consideration" for acquisition. It also, obviously,  helps the selection focus in the real candidates without debating whether a solution which falls short (but which some members like) should be a finalist or not.

Obviously, Gartner clients who want to understand these points (and others for which I cannot blog) better, will want to see the toolkit RFI questions,and selection criteria and ratings spreadsheet.

*Available to Gartner clients or for a fee

Submit a Comment »

Category: Uncategorized     Tags:

Share