Empirical Investigation of Optimization Algorithms in Neural Machine Translation
The Prague Bulletin of Mathematical Linguistics2017Vol. 108(1), pp. 13–25
Citations Over TimeTop 10% of 2017 papers
Abstract
Abstract Training neural networks is a non-convex and a high-dimensional optimization problem. In this paper, we provide a comparative study of the most popular stochastic optimization techniques used to train neural networks. We evaluate the methods in terms of convergence speed, translation quality, and training stability. In addition, we investigate combinations that seek to improve optimization in terms of these aspects. We train state-of-the-art attention-based models and apply them to perform neural machine translation. We demonstrate our results on two tasks: WMT 2016 En→Ro and WMT 2015 De→En.
Related Papers
- Pre-Translation for Neural Machine Translation.(2016)
- → Evaluation of English–Slovak Neural and Statistical Machine Translation(2021)21 cited
- → Machine Translation Using Deep Learning: A Comparison(2020)4 cited
- → The development and advance of machine translation(2023)1 cited
- → Research on machine analogy translation technology based on machine learning(2023)