Probabilistic Top-Down Parsing and Language Modeling

Citations Over TimeTop 1% of 2001 papers

Abstract

This paper describes the functioning of a broad-coverage probabilistic top-down parser, and its application to the problem of language modeling for speech recognition. The paper first introduces key notions in language modeling and probabilistic parsing, and briefly reviews some previous approaches to using syntactic structure for language modeling. A lexicalized probabilistic top-down parser is then presented, which performs very well, in terms of both the accuracy of returned parses and the efficiency with which they are found, relative to the best broad-coverage statistical parsers. A new language model that utilizes probabilistic top-down parsing is then outlined, and empirical results show that it improves upon previous work in test corpus perplexity. Interpolation with a trigram model yields an exceptional improvement relative to the improvement observed by other models, demonstrating the degree to which the information captured by our parsing model is orthogonal to that captured by a trigram model. A small recognition experiment also demonstrates the utility of the model.

Related Papers

• → Class phrase models for language modeling(2002)45 cited
• → Decoupling Word-Pair Distance and Co-occurrence Information for Effective Long History Context Language Modeling(2015)2 cited
• → Improving language models by using distant information(2007)3 cited
• → Verifying the long-range dependency of RNN language models(2016)2 cited
• → Recognition Performance of a Structured Language Model(2000)32 cited