Evaluating the Effectiveness of Personalized Web Search

Citations Over TimeTop 10% of 2008 papers

Abstract

Although personalized search has been under way for many years and many personalization algorithms have been investigated, it is still unclear whether personalization is consistently effective on different queries for different users and under different search contexts. In this paper, we study this problem and provide some findings. We present a large-scale evaluation framework for personalized search based on query logs and then evaluate five personalized search algorithms (including two click-based ones and three topical-interest-based ones) using 12-day query logs of Windows Live Search. By analyzing the results, we reveal that personalized Web search does not work equally well under various situations. It represents a significant improvement over generic Web search for some queries, while it has little effect and even harms query performance under some situations. We propose click entropy as a simple measurement on whether a query should be personalized. We further propose several features to automatically predict when a query will benefit from a specific personalization algorithm. Experimental results show that using a personalization algorithm for queries selected by our prediction model is better than using it simply for all queries.

Related Papers

• → When do people use query suggestion? A query suggestion log analysis(2013)41 cited
• → Log mining to support web query expansions(2009)6 cited
• → Query Expansion for Improving Web Search(2020)4 cited
• → Interactive Web Multimedia Search Using Query-Session-Based Query Expansion(2001)1 cited
• → Query Recommendation Using Hybrid Query Relevance(2018)1 cited