Blog post

How Do We Know Search Is Working?

By Whit Andrews | May 30, 2014 | 0 Comments

A client called this week to ask how she could know enterprise search is working. The CIO wants to know that the company is truly a leader in its search, and to do that, he feels that there need to be solid measurements he can see that show it’s definitely getting better. Even more attractive would be analytics that show it’s better for his company than it is for competitors.

There are lots of ways to tell. I frequently reread Marti Hearst’s delightful, lovely Search User Interfaces to remind myself of some. There’s a whole CHAPTER on “How To Evaluate Search User Interfaces.”  (REALLY. IKR? You are as excited about this as I am. And it’s free online. I read it on paper. Don’t hate me because I’m a Luddite.)

Les chiens des deux chasseurs

Hunting dogs near my friend Jeff Mann’s home in France. They’re HUNTING. Get it? HUNTING. It’s a RELATED IMAGE.

Also, I ran into this intriguing blog entry today from Jaime Teevan today via twitter. (I wish I knew whom to h/t; I can’t find the tweet again in the barrage of  interest around some dude who bought a basketball team.) The idea in a study was to send users on scavenger hunts to find the answers to their questions; if they got the answer right, it meant the search worked.  In this case, it was mTurk work on public testing, but it makes sense in organizations as well. Frequently, key stakeholders are upset about particular searches, which could certainly serve as straw men to knock down. A/B testing is too rare in such situations; a 5-question test might be definitive when selecting between a few options for interfaces.

This is handy; I’ll start suggesting it right away.

Comments are closed