Generating Sentences by Editing Prototypes
Transactions of the Association for Computational Linguistics2018Vol. 6, pp. 437–450
Citations Over TimeTop 1% of 2018 papers
Abstract
We propose a new generative language model for sentences that first samples a prototype sentence from the training corpus and then edits it into a new sentence. Compared to traditional language models that generate from scratch either left-to-right or by first sampling a latent sentence vector, our prototype-then-edit model improves perplexity on language modeling and generates higher quality outputs according to human evaluation. Furthermore, the model gives rise to a latent edit vector that captures interpretable semantics such as sentence similarity and sentence-level analogies.
Related Papers
- Combination of Recurrent Neural Networks and Factored Language Models for Code-Switching Language Modeling(2013)
- → Improved topic-dependent language modeling using information retrieval techniques(1999)55 cited
- → Verifying the long-range dependency of RNN language models(2016)2 cited
- → Attention-based Persian Language Modeling(2022)
- → Building Personalized Language Models Through Language Model Interpolation(2023)