AirDialogue: An Environment for Goal-Oriented Dialogue Research

Citations Over TimeTop 10% of 2018 papers

Abstract

Recent progress in dialogue generation has inspired a number of studies on dialogue systems that are capable of accomplishing tasks through natural language interactions. A promising direction among these studies is the use of reinforcement learning techniques, such as self-play, for training dialogue agents. However, current datasets are limited in size, and the environment for training agents and evaluating process is relatively unsophisticated. We present AirDialogue, a large dataset that contains 402,038 goal-oriented conversations. To collect this dataset, we create a contextgenerator which provides travel and flight restrictions. We then ask human annotators to play the role of a customer or an agent and interact with the goal of successfully booking a trip given the restrictions. Key to our environment is the ease of evaluating the success of the dialogue, which is achieved by using ground-truth states (e.g., the flight being booked) generated by the restrictions. Any dialogue agent that does not generate the correct states is considered to fail. Our experimental results indicate that state-of-the-art dialogue models on the test dataset can only achieve a scaled score of 0.22 and an exact match score of 0.1 while humans can reach a score of 0.94 and 0.93 respectively, which suggests significant opportunities for future improvement.

Related Papers

• → Task offloading method of edge computing in internet of vehicles based on deep reinforcement learning(2022)112 cited
• → Intelligent ubiquitous computing for future UAV-enabled MEC network systems(2021)59 cited
• → Reinforcement Learning Based on Contextual Bandits for Personalized Online Learning Recommendation Systems(2020)45 cited
• → Offloading in Mobile Edge Computing Based on Federated Reinforcement Learning(2022)7 cited
• → Asking Data in a Controlled Way with Ask Data Anything NQL(2016)1 cited