Improving Real-time Recognition of Morphologically Rich Speech with Transformer Language Model
Citations Over Time
Abstract
Transformer models have become to state-of-the-art in natural language understanding, their use for language modeling in Automatic Speech Recognition (ASR) is also promising. Albeit Transformer based language models were shown to improve ASR performance, their computational complexity makes their application in real-time systems quite challenging. It has been also shown that the knowledge of such language models can be transferred to traditional n-gram models, suitable for real-time decoding. This paper investigates the adaptation of this transfer approach to morphologically rich languages, and in a real time scenario. We propose a new method for subword-based neural text augmentation with a Transformer language model, which consists in retokenizing the training corpus into subwords, based on a statistical data-driven approach. We demonstrate that ASR performance can be augmented by yet reducing the vocabulary size and alleviating memory consumption.
Related Papers
- → Language Model Adaptation For Statistical Machine Translation Based On Information Retrieval(2004)77 cited
- → Random forests and the data sparseness problem in language modeling(2006)62 cited
- Code-Switch Language Model with Inversion Constraints for Mixed Language Speech Recognition(2012)
- → An empirical study of statistical language models: n-gram language models vs. neural network language models(2018)10 cited
- → Trends and challenges in language modeling for speech recognition and machine translation(2009)3 cited