I had a fun vendor briefing with Brainspace today. Fun, because I’m old now, so it was great to meet a founder who developed his knowledge of search in the Verity era (he built “topics,” for you other grizzled or afflicted folks out there). And fun, because we talked about transparency of relevancy modeling, an important topic in our evaluation of vendors in search, content analytics, and support for knowledge management.
What I mean in the search world by “transparency” is this: Can an administrator, or in an even better case an end user, determine why a particular result set is chosen? It’s a grail, and it’s an asymptote. In the old days, before term frequency/inverse document frequency descended to the Earth and made it possible to rank results meaningfully, “relevancy” mean “hits.” My professor in college showed us that one could filter Moby Dick to see the appearances of a given word or phrase. (“Ah, the world, oh, the whale.” Ah, the DOSfish, indeed.) In such a world, “Moby Dick” is a possible result for “whale” or “Ishmael” or “ship.” No value judgment appeared.
Since then, many such values have emerged, around elements such as term frequency and with great sophistication, including statistical analysis. One vendor was fond of pointing out to me that if the vast majority of users couldn’t understand how it worked, it was also unreasonable (and possibly really mean) to expect that vast majority to understand how it reached a given result set. Another vendor points out that its key advantage is how it works, so revealing that is a bad idea, sort of like Coca Cola writing its formula on the side of the iconic bottle.
Brainspace — not in the current enterprise search search MQ I am putting together, as it does not meet all the criteria — believes in transparency and avoiding the “black box” model of “This is what we gave you; like it or leave it.” They’re still developing a profile in the market, of course.
What transparency allows is for improvement. When information architects can see how results are achieved, they can also improve them more effectively. It’s like the tab in Pandora that allows you to see why certain songs are being played for you, or in Netflix to see why you’re being offered some particular movies. (Currently, I am being recommended “Mad Men,” which I have tried twice…because I liked “Fergo”? Mmm…k.) It’s like the window in the aquarium to which I link above — by seeing what the query engine does, you can put yourself in the place of a fish to understand the dynamics of the tank.
And it’s fundamental. While transparency in other areas can be a buzzword, in search it’s of great value. Without transparency, authority depends only on faith, which fails in information architecture.
The Gartner Blog Network provides an opportunity for Gartner analysts to test ideas and move research forward. Because the content posted by Gartner analysts on this site does not undergo our standard editorial review, all comments or opinions expressed hereunder are those of the individual contributors and do not represent the views of Gartner, Inc. or its management.