Searching for answers about search? Expand your understanding of search engines with the first video in our series on evaluating search quality.
TRANSCRIPT:
Hi, I’m Suvda Myagmar. I’m a software engineer at Expect Labs. In this video I will talk about evaluating search quality. Search evaluation is the process of measuring the relevance of search results and the quality of users’ experience.
There are no industry-wide accepted methods or accepted metrics that measure the quality of search results, but there are common practices. I will talk about common practices based on my experience. I’ve worked at Yahoo Search, Microsoft Bing Search, and currently I’m in charge of search quality at Expect Labs.
How do you know whether your search engine is performing well relative to other search engines, or whether the search quality improved after you’ve modified your ranking algorithm?
Search ranking algorithms are often measured by metrics such as precision, recall, accuracy, F-measure, and NDCG. The data necessary to compute these metrics can be collected using live A/B testing or manual rating by human judges. Next, I will talk about these search metrics and data collection methods in more detail.
