Selecting Informative Contexts Improves Language Model Fine-tuning

Citations Over Time

Abstract

Language model fine-tuning is essential for modern natural language processing, but is computationally expensive and time-consuming. Further, the effectiveness of fine-tuning is limited by the inclusion of training examples that negatively affect performance. Here we present a general fine-tuning method that we call information gain filtration for improving the overall training efficiency and final performance of language model fine-tuning. We define the information gain of an example as the improvement on a test metric after training on that example. A secondary learner is then trained to approximate this quantity. During fine-tuning, this learner selects informative examples and skips uninformative ones. We show that our method has consistent improvement across datasets, fine-tuning tasks, and language model architectures. For example, we achieve a median perplexity of 54.0 on a books dataset compared to 57.3 for standard fine-tuning. We present statistical evidence that offers insight into the improvements of our method over standard fine-tuning. The generality of our method leads us to propose a new paradigm for language model fine-tuning -- we encourage researchers to release pretrained secondary learners on common corpora to promote efficient and effective fine-tuning, thereby improving the performance and reducing the overall energy footprint of language model fine-tuning.

Related Papers

• → Universal Language Model Fine-tuning for Text Classification(2018)340 cited
• → Tri-Train: Automatic Pre-Fine Tuning between Pre-Training and Fine-Tuning for SciNER(2020)10 cited
• → Making Pre-trained Language Models Better Few-shot Learners(2020)128 cited
• → Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning(2020)60 cited