Mixed Negative Sampling for Learning Two-tower Neural Networks in Recommendations

Citations Over TimeTop 10% of 2020 papers

Abstract

Learning query and item representations is important for building large scale recommendation systems. In many real applications where there is a huge catalog of items to recommend, the problem of efficiently retrieving top k items given user’s query from deep corpus leads to a family of factorized modeling approaches where queries and items are jointly embedded into a low-dimensional space. In this paper, we first showcase how to apply a two-tower neural network framework, which is also known as dual encoder in the natural language community, to improve a large-scale, production app recommendation system. Furthermore, we offer a novel negative sampling approach called Mixed Negative Sampling (MNS). In particular, different from commonly used batch or unigram sampling methods, MNS uses a mixture of batch and uniformly sampled negatives to tackle the selection bias of implicit user feedback. We conduct extensive offline experiments using large-scale production dataset and show that MNS outperforms other baseline sampling methods. We also conduct online A/B testing and demonstrate that the two-tower retrieval model based on MNS significantly improves retrieval quality by encouraging more high-quality app installs.

Related Papers

• → Physician-Friendly Machine Learning: A Case Study with Cardiovascular Disease Risk Prediction(2019)71 cited
• → Application of Machine Learning in Animal Disease Analysis and Prediction(2020)26 cited
• → Interpretable machine learning assessment(2023)24 cited
• → Breakdown of Machine Learning Algorithms(2022)1 cited
• → Machine Learning Techniques for the Management of Diseases: A Paper Review(2024)