RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses

Citations Over TimeTop 10% of 2023 papers

Abstract

Pretrained language models such as BERT have been shown to be exceptionally effective for text ranking. However, there are limited studies on how to leverage more powerful sequence-to-sequence models such as T5. Existing attempts usually formulate text ranking as a classification problem and rely on postprocessing to obtain a ranked list. In this paper, we propose RankT5 and study two T5-based ranking model structures, an encoder-decoder and an encoder-only one, so that they not only can directly output ranking scores for each query-document pair, but also can be fine-tuned with pairwise or listwise ranking losses to optimize ranking performance. Our experiments show that the proposed models with ranking losses can achieve substantial ranking performance gains on different public text ranking data sets. Moreover, ranking models fine-tuned with listwise ranking losses have better zero-shot ranking performance on out-of-domain data than models fine-tuned with classification losses.

Related Papers

• → QoRank: A query-dependent ranking model using LSE-based weighted multiple hyperplanes aggregation for information retrieval(2010)7 cited
• → Querywise Fair Learning to Rank through Multi-Objective Optimization(2023)5 cited
• → Bi-Rank: A New Bi-Directional Ranking Method for Goods Selection(2020)1 cited
• → OrdRank: Learning to Rank with Ordered Multiple Hyperplanes(2009)
• Survey of information retrieval ranking algorithm(2009)