Generating Sentences by Editing Prototypes

Citations Over TimeTop 1% of 2018 papers

Abstract

We propose a new generative language model for sentences that first samples a prototype sentence from the training corpus and then edits it into a new sentence. Compared to traditional language models that generate from scratch either left-to-right or by first sampling a latent sentence vector, our prototype-then-edit model improves perplexity on language modeling and generates higher quality outputs according to human evaluation. Furthermore, the model gives rise to a latent edit vector that captures interpretable semantics such as sentence similarity and sentence-level analogies.

Related Papers

• Combination of Recurrent Neural Networks and Factored Language Models for Code-Switching Language Modeling(2013)
• → Improved topic-dependent language modeling using information retrieval techniques(1999)55 cited
• → Verifying the long-range dependency of RNN language models(2016)2 cited
• → Attention-based Persian Language Modeling(2022)
• → Building Personalized Language Models Through Language Model Interpolation(2023)