BERTje: A Dutch BERT Model
arXiv (Cornell University)2019
Citations Over Time
Wietse de Vries, Andreas van Cranenburgh, Arianna Bisazza, Tommaso Caselli, Gertjan van Noord, Malvina Nissim
Abstract
The transformer-based pre-trained language model BERT has helped to improve state-of-the-art performance on many natural language processing (NLP) tasks. Using the same architecture and parameters, we developed and evaluated a monolingual Dutch BERT model called BERTje. Compared to the multilingual BERT model, which includes Dutch but is only based on Wikipedia text, BERTje is based on a large and diverse dataset of 2.4 billion tokens. BERTje consistently outperforms the equally-sized multilingual BERT model on downstream NLP tasks (part-of-speech tagging, named-entity recognition, semantic role labeling, and sentiment analysis). Our pre-trained Dutch BERT model is made available at https://github.com/wietsedv/bertje.
Related Papers
- → Evaluating Pretrained Transformer-based Models on the Task of Fine-Grained Named Entity Recognition(2020)31 cited
- → A Comparative Study of Dictionary-based and Machine Learning-based Named Entity Recognition in Pashto(2020)10 cited
- → Named Entity Recognition: A Survey for Indian Languages(2019)16 cited
- → Studying the impact of various features on the performance of Conditional Random Field-based Arabic Named Entity Recognition(2013)5 cited
- → Optimization Strategies for BERT-Based Named Entity Recognition(2023)1 cited