Position Bias Estimation for Unbiased Learning to Rank in Personal Search

Citations Over TimeTop 1% of 2018 papers

Abstract

A well-known challenge in learning from click data is its inherent bias and most notably position bias. Traditional click models aim to extract the ‹query, document› relevance and the estimated bias is usually discarded after relevance is extracted. In contrast, the most recent work on unbiased learning-to-rank can effectively leverage the bias and thus focuses on estimating bias rather than relevance [20, 31]. Existing approaches use search result randomization over a small percentage of production traffic to estimate the position bias. This is not desired because result randomization can negatively impact users' search experience. In this paper, we compare different schemes for result randomization (i.e., RandTopN and RandPair) and show their negative effect in personal search. Then we study how to infer such bias from regular click data without relying on randomization. We propose a regression-based Expectation-Maximization (EM) algorithm that is based on a position bias click model and that can handle highly sparse clicks in personal search. We evaluate our EM algorithm and the extracted bias in the learning-to-rank setting. Our results show that it is promising to extract position bias from regular clicks without result randomization. The extracted bias can improve the learning-to-rank algorithms significantly. In addition, we compare the pointwise and pairwise learning-to-rank models. Our results show that pairwise models are more effective in leveraging the estimated bias.

Related Papers

• → Necessary and sufficient optimality conditions for elliptic control problems with finitely many pointwise state constraints(2007)20 cited
• → The Pointwise Estimates of Solutions for Semilinear Dissipative Wave Equation(2008)6 cited
• → On pointwise Kan extensions in double categories(2014)6 cited
• → On bi-pointwise control of a wave equation and algorithm(2000)
• → Filtering and Control under Pointwise in Time Bounds by the Use of Central Solutions(1995)