Is word error rate a good indicator for spoken language understanding accuracy

Citations Over TimeTop 10% of 2004 papers

Abstract

It is a conventional wisdom in the speech community that better speech recognition accuracy is a good indicator for better spoken language understanding accuracy, given a fixed understanding component. The findings in this work reveal that this is not always the case. More important than word error rate reduction, the language model for recognition should be trained to match the optimization objective for understanding. In this work, we applied a spoken language understanding model as the language model in speech recognition. The model was obtained with an example-based learning algorithm that optimized the understanding accuracy. Although the speech recognition word error rate is 46% higher than the trigram model, the overall slot understanding error can be reduced by as much as 17%.

Related Papers

• → A dynamic language model for speech recognition(1991)126 cited
• → Class phrase models for language modeling(2002)45 cited
• → Speech recognition using a stochastic language model integrating local and global constraints(1994)9 cited
• → Decoupling Word-Pair Distance and Co-occurrence Information for Effective Long History Context Language Modeling(2015)2 cited
• → Improving language models by using distant information(2007)3 cited