He Thinks He Knows Better than the Doctors: BERT for Event Factuality Fails on Pragmatics
Transactions of the Association for Computational Linguistics2021Vol. 9, pp. 1081–1097
Citations Over TimeTop 20% of 2021 papers
Abstract
Abstract We investigate how well BERT performs on predicting factuality in several existing English datasets, encompassing various linguistic constructions. Although BERT obtains a strong performance on most datasets, it does so by exploiting common surface patterns that correlate with certain factuality labels, and it fails on instances where pragmatic reasoning is necessary. Contrary to what the high performance suggests, we are still far from having a robust system for factuality prediction.
Related Papers
- → Pragmatics and Instructional Pragmatics in Second/Foreign Language Preservice Teacher Education: A Case Study in Australia and Vietnam(2023)9 cited
- → Pragmatics and Dialogue(2012)11 cited
- → Introduction(2016)4 cited
- → Assessment of Pragmatics(2012)7 cited
- Recent Developments in Pragmatics as Evidenced by Handbook of Pragmatics(2008)