Long short-term memory recurrent neural network architectures for large scale acoustic modeling

Citations Over TimeTop 1% of 2014 papers

Abstract

Long Short-Term Memory (LSTM) is a specific recurrent neural network (RNN) architecture that was designed to model temporal sequences and their long-range dependencies more accurately than conventional RNNs. In this paper, we explore LSTM RNN architectures for large scale acoustic modeling in speech recognition. We recently showed that LSTM RNNs are more effective than DNNs and conventional RNNs for acoustic modeling, considering moderately-sized models trained on a single machine. Here, we introduce the first distributed training of LSTM RNNs using asynchronous stochastic gradient descent optimization on a large cluster of machines. We show that a two-layer deep LSTM RNN where each LSTM layer has a linear recurrent projection layer can exceed state-of-the-art speech recognition performance. This architecture makes more effective use of model parameters than the others considered, converges quickly, and outperforms a deep feed forward neural network having an order of magnitude more parameters. Index Terms: Long Short-Term Memory, LSTM, recurrent neural network, RNN, speech recognition, acoustic modeling.

Related Papers

• → Vowel based Voice Activity Detection with LSTM Recurrent Neural Network(2016)25 cited
• → Artificial Intelligence for Sport Actions and Performance Analysis using Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM)(2018)10 cited
• → Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent\n Neural Networks(2019)497 cited
• → Accident Detection System Based on RNN Exploiting Keypoints and LSTM(2023)
• → Hoax Identification On Social Media Using Recurrent Neural Network (RNN) And Long Short-term Memory (LSTM) Methods(2023)