Grammar Induction with Neural Language Models: An Unusual Replication
Citations Over TimeTop 10% of 2018 papers
Abstract
A substantial thread of recent work on latent tree learning has attempted to develop neural network models with parse-valued latent variables and train them on non-parsing tasks, in the hope of having them discover interpretable tree structure. In a recent paper, Shen et al. ( In an attempt to reproduce these results, we discover issues that make the original results hard to trust, including tuning and even training on what is effectively the test set. Here, we attempt to reproduce these results in a fair experiment and to extend them to two new datasets. We find that the results of this work are robust: All variants of the model under study outperform all latent tree learning baselines, and perform competitively with symbolic grammar induction systems. We find that this model represents the first empirical success for latent tree learning, and that neural network language modeling warrants further study as a setting for grammar induction.
Related Papers
- → Parse Forest Diagnostics with Dr. Ambiguity(2012)9 cited
- → Production trees: a compact representation of parsed programs(1990)5 cited
- Compact storage of syntax trees for ambiguous sentences(1992)
- → Browsing the Parse Space(2015)
- → Parse Trees(2019)