Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics

Citations Over TimeTop 10% of 2021 papers

Abstract

Much of recent progress in NLU was shown to be due to models' learning dataset-specific heuristics. We conduct a case study of generalization in NLI (from MNLI to the adversarially constructed HANS dataset) in a range of BERT-based architectures (adapters, Siamese Transformers, HEX debiasing), as well as with subsampling the data and increasing the model size. We report 2 successful and 3 unsuccessful strategies, all providing insights into how Transformer-based models learn to generalize.

Related Papers

• → Active Learning Based on New Localized Generalization Error Model for Training RBFNN(2010)2 cited
• → More Communication Does Not Result in Smaller Generalization Error in Federated Learning(2023)2 cited
• → Learning in a large committee machine: worst case and average case(1996)8 cited
• The bounds of change-one generalization error under united algorithmic(2005)
• → More Communication Does Not Result in Smaller Generalization Error in Federated Learning(2023)