Improved Noisy Student Training for Automatic Speech Recognition

Citations Over TimeTop 1% of 2020 papers

Abstract

Recently, a semi-supervised learning method known as "noisy student training" has been shown to improve image classification performance of deep networks significantly. Noisy student training is an iterative self-training method that leverages augmentation to improve network performance. In this work, we adapt and improve noisy student training for automatic speech recognition, employing (adaptive) SpecAugment as the augmentation method. We find effective methods to filter, balance and augment the data generated in between self-training iterations. By doing so, we are able to obtain word error rates (WERs) 4.2%/8.6% on the clean/noisy LibriSpeech test sets by only using the clean 100h subset of LibriSpeech as the supervised set and the rest (860h) as the unlabeled set. Furthermore, we are able to achieve WERs 1.7%/3.4% on the clean/noisy LibriSpeech test sets by using the unlab-60k subset of LibriLight as the unlabeled set for LibriSpeech 960h. We are thus able to improve upon the previous state-of-the-art clean/noisy test WERs achieved on LibriSpeech 100h (4.74%/12.20%) and LibriSpeech (1.9%/4.1%).

Related Papers

• → Training Systems Concept for the Armored Family of Vehicles with Consideration of the Roles of Embedded Training and Stand-Alone Training Devices(1988)1 cited
• Hydroelectric Construction Jobs and related Training | Hydro Northern Training Initiative | Manitoba Competitiveness, Training and Trade(2004)
• 介護実習IIにおける実習計画表の活用の検討 : 個別援助技術実習と介護総合実習との比較 ; Consideration of utilization of a training agenda in care practical training II － The comparative of individual assistance technical training and total practice training －(2016)
• Effects of Training Input on Training Performance of Airline Service Training - Focused on Mediating Role of Training Process -(2012)