The Effect of Domain and Diacritics in Yorùbá-English Neural Machine Translation

Citations Over TimeTop 12% of 2021 papers

Abstract

Massively multilingual machine translation (MT) has shown impressive capabilities, including zero and few-shot translation between low-resource language pairs. However, these models are often evaluated on high-resource languages with the assumption that they generalize to low-resource ones. The difficulty of evaluating MT models on low-resource pairs is often due to lack of standardized evaluation datasets. In this paper, we present MENYO-20k, the first multi-domain parallel corpus with a special focus on clean orthography for Yorùbá--English with standardized train-test splits for benchmarking. We provide several neural MT benchmarks and compare them to the performance of popular pre-trained (massively multilingual) MT models both for the heterogeneous test set and its subdomains. Since these pre-trained models use huge amounts of data with uncertain quality, we also analyze the effect of diacritics, a major characteristic of Yorùbá, in the training data. We investigate how and when this training condition affects the final quality and intelligibility of a translation. Our models outperform massively multilingual models such as Google ($+8.7$ BLEU) and Facebook M2M ($+9.1$ BLEU) when translating to Yorùbá, setting a high quality benchmark for future research.

Related Papers

• Identifying bilingual Multi-Word Expressions for Statistical Machine Translation(2012)
• Semantical and Syntactical Analysis of NLP(2014)
• → Towards State-of-the-art English-Vietnamese Neural Machine Translation(2017)8 cited
• → English-Japanese Neural Machine Translation with Encoder-Decoder-Reconstructor(2017)2 cited
• → Recurrent Stacking of Layers for Compact Neural Machine Translation Models(2018)2 cited