Portuguese Named Entity Recognition using BERT-CRF

Citations Over Time

Abstract

Recent advances in language representation using neural networks have made it viable to transfer the learned internal states of a trained model to downstream natural language processing tasks, such as named entity recognition (NER) and question answering. It has been shown that the leverage of pre-trained language models improves the overall performance on many tasks and is highly beneficial when labeled data is scarce. In this work, we train Portuguese BERT models and employ a BERT-CRF architecture to the NER task on the Portuguese language, combining the transfer capabilities of BERT with the structured predictions of CRF. We explore feature-based and fine-tuning training strategies for the BERT model. Our fine-tuning approach obtains new state-of-the-art results on the HAREM I dataset, improving the F1-score by 1 point on the selective scenario (5 NE classes) and by 4 points on the total scenario (10 NE classes).

Related Papers

• → Evaluation of clinical named entity recognition methods for Serbian electronic health records(2022)16 cited
• → Chinese Medical Named Entity Recognition using CRF-MT-Adapt and NER-MRC(2021)11 cited
• → Optimization Strategies for BERT-Based Named Entity Recognition(2023)1 cited
• → Transfer Learning for Domain-Specific Named Entity Recognition in German(2020)