Visual analysis of attention-based end-to-end speech recognition

Citations Over Time

Abstract

An end-to-end speech recognition model consisting of a single integrated neural network model was recently proposed. The end-to-end model does not need several training steps, and its structure is easy to understand. However, it is difficult to understand how the model recognizes speech internally. In this paper, we visualized and analyzed the attention-based end-to-end model to elucidate its internal mechanisms. We compared the acoustic model of the BLSTM-HMM hybrid model with the encoder of the end-to-end model, and visualized them using t-SNE to examine the difference between neural network layers. As a result, we were able to delineate the difference between the acoustic model and the end-to-end model encoder. Additionally, we analyzed the decoder of the end-to-end model from a language model perspective. Finally, we found that improving end-to-end model decoder is necessary to yield higher performance.

Related Papers

• → Indonesian Automatic Speech Recognition system using CMUSphinx toolkit and limited dataset(2016)19 cited
• → Hybrid CTC-Attention Network-Based End-to-End Speech Recognition System for Korean Language(2022)4 cited
• → A Continuous Speech Recognition System for Bangla Language(2021)1 cited
• → Improved DNN-HMM English Acoustic Model Specially For Phonotactic Language Recognition(2019)
• → Overview of Speech Recognition with Privacy(2012)