Towards a Human-like Open-Domain Chatbot
Citations Over Time
Abstract
We present Meena, a multi-turn open-domain chatbot trained end-to-end on data mined and filtered from public domain social media conversations. This 2.6B parameter neural network is simply trained to minimize perplexity of the next token. We also propose a human evaluation metric called Sensibleness and Specificity Average (SSA), which captures key elements of a human-like multi-turn conversation. Our experiments show strong correlation between perplexity and SSA. The fact that the best perplexity end-to-end trained Meena scores high on SSA (72% on multi-turn evaluation) suggests that a human-level SSA of 86% is potentially within reach if we can better optimize perplexity. Additionally, the full version of Meena (with a filtering mechanism and tuned decoding) scores 79% SSA, 23% higher in absolute SSA than the existing chatbots we evaluated.
Related Papers
- → HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanities Use Case(2019)17,204 cited
- → Personalizing Dialogue Agents: I have a dog, do you have pets too?(2018)1,170 cited
- → DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation(2020)1,040 cited
- Wizard of Wikipedia: Knowledge-Powered Conversational Agents(2018)
- → Recipes for Building an Open-Domain Chatbot(2021)171 cited