Pre-trained models, data augmentation, and ensemble learning for biomedical information extraction and document classification

Citations Over TimeTop 25% of 2022 papers

Abstract

Large volumes of publications are being produced in biomedical sciences nowadays with ever-increasing speed. To deal with the large amount of unstructured text data, effective natural language processing (NLP) methods need to be developed for various tasks such as document classification and information extraction. BioCreative Challenge was established to evaluate the effectiveness of information extraction methods in biomedical domain and facilitate their development as a community-wide effort. In this paper, we summarize our work and what we have learned from the latest round, BioCreative Challenge VII, where we participated in all five tracks. Overall, we found three key components for achieving high performance across a variety of NLP tasks: (1) pre-trained NLP models; (2) data augmentation strategies and (3) ensemble modelling. These three strategies need to be tailored towards the specific tasks at hands to achieve high-performing baseline models, which are usually good enough for practical applications. When further combined with task-specific methods, additional improvements (usually rather small) can be achieved, which might be critical for winning competitions. Database URL: https://doi.org/10.1093/database/baac066.

Related Papers

• → Accounting for baseline targets in NDCs(2018)3 cited
• Method for Improvement of Baseline Resolving Quality in GPS Measurement(2007)
• Analysis of the Baseline Decorrelation and Critical Baseline of Interferometric SAR(2003)
• Research of Baseline Implement(2010)
• Study and application of telecom operators system security state baseline(2012)