Blog post

How Do We Know Search Is Working?

By Whit Andrews | May 30, 2014 | 0 Comments

A client called this week to ask how she could know enterprise search is working. The CIO wants to know that the company is truly a leader in its search, and to do that, he feels that there need to be solid measurements he can see that show it’s definitely getting better. Even more attractive would be analytics that show it’s better for his company than it is for competitors.

There are lots of ways to tell. I frequently reread Marti Hearst’s delightful, lovely Search User Interfaces to remind myself of some. There’s a whole CHAPTER on “How To Evaluate Search User Interfaces.”  (REALLY. IKR? You are as excited about this as I am. And it’s free online. I read it on paper. Don’t hate me because I’m a Luddite.)

Les chiens des deux chasseurs

Hunting dogs near my friend Jeff Mann’s home in France. They’re HUNTING. Get it? HUNTING. It’s a RELATED IMAGE.

Also, I ran into this intriguing blog entry today from Jaime Teevan today via twitter. (I wish I knew whom to h/t; I can’t find the tweet again in the barrage of  interest around some dude who bought a basketball team.) The idea in a study was to send users on scavenger hunts to find the answers to their questions; if they got the answer right, it meant the search worked.  In this case, it was mTurk work on public testing, but it makes sense in organizations as well. Frequently, key stakeholders are upset about particular searches, which could certainly serve as straw men to knock down. A/B testing is too rare in such situations; a 5-question test might be definitive when selecting between a few options for interfaces.

This is handy; I’ll start suggesting it right away.

The Gartner Blog Network provides an opportunity for Gartner analysts to test ideas and move research forward. Because the content posted by Gartner analysts on this site does not undergo our standard editorial review, all comments or opinions expressed hereunder are those of the individual contributors and do not represent the views of Gartner, Inc. or its management.

Comments are closed