Spoken language understanding using long short-term memory neural networks

Citations Over TimeTop 1% of 2014 papers

Abstract

Neural network based approaches have recently produced record-setting performances in natural language understanding tasks such as word labeling. In the word labeling task, a tagger is used to assign a label to each word in an input sequence. Specifically, simple recurrent neural networks (RNNs) and convolutional neural networks (CNNs) have shown to significantly outperform the previous state-of-the-art - conditional random fields (CRFs). This paper investigates using long short-term memory (LSTM) neural networks, which contain input, output and forgetting gates and are more advanced than simple RNN, for the word labeling task. To explicitly model output-label dependence, we propose a regression model on top of the LSTM un-normalized scores. We also propose to apply deep LSTM to the task. We investigated the relative importance of each gate in the LSTM by setting other gates to a constant and only learning particular gates. Experiments on the ATIS dataset validated the effectiveness of the proposed models.

Related Papers

• → Recurrent conditional random field for language understanding(2014)112 cited
• → Hybrid semi-Markov CRF for Neural Sequence Labeling(2018)56 cited
• → Research of Applying Chain Conditional Random Fields to Semantic Role Labeling(2009)1 cited
• → Hybrid semi-Markov CRF for Neural Sequence Labeling(2018)3 cited
• → A Novel Method of Citation Sequence Labeling Based on Conditional Random Fields(2013)