Language Model Pre-training Method in Machine Translation Based on Named Entity Recognition

Citations Over TimeTop 17% of 2020 papers

Abstract

Neural Machine Translation (NMT) model has become the mainstream technology in machine translation. The supervised neural machine translation model trains with abundant of sentence-level parallel corpora. But for low-resources language or dialect with no such corpus available, it is difficult to achieve good performance. Researchers began to focus on unsupervised neural machine translation (UNMT) that monolingual corpus as training data. UNMT need to construct the language model (LM) which learns semantic information from the monolingual corpus. This paper focuses on the pre-training of LM in unsupervised machine translation and proposes a pre-training method, NER-MLM (named entity recognition masked language model). Through performing NER, the proposed method can obtain better semantic information and language model parameters with better training results. In the unsupervised machine translation task, the BLEU scores on the WMT’16 English–French, English–German, data sets are 35.30, 27.30 respectively. To the best of our knowledge, this is the highest results in the field of UNMT reported so far.

Related Papers

• → Neural Machine Translation for English to Hindi(2018)52 cited
• → Simplification of English and Bengali Sentences for Improving Quality of Machine Translation(2022)15 cited
• → Statistical Machine Translation(2010)1 cited
• → Automatic Post-Editing Method Using Translation Knowledge Based on Intuitive Common Parts Continuum for Statistical Machine Translation(2014)
• The Exploring on Automatic Machine Translation Evaluation with Sentence Structure Information(2010)