Learning through Dialogue Interactions by Asking Questions

A good dialogue agent should have the ability to interact with users by both responding to questions and by asking questions, and importantly to learn from both types of interaction. In this work, we explore this direction by designing a simulator and a set of synthetic tasks in the movie domain that allow such interactions between a learner and a teacher. We investigate how a learner can benefit from asking questions in both offline and online reinforcement learning settings, and demonstrate that the learner improves when asking questions. Finally, real experiments with Mechanical Turk validate the approach. Our work represents a first step in developing such end-to-end learned interactive dialogue agents.

[1]  Terry Winograd,et al.  Understanding natural language , 1974 .

[2]  R. Houten.  Learning through feedback , 1980 .

[3]  M. Werts,et al.  Instructive feedback: Review of parameters and effects , 1995 .

[4]  A. Latham.  Learning through Feedback. , 1997 .

[5]  R. Higgins,et al.  The Conscientious Consumer: Reconsidering the role of assessment feedback in student learning , 2002 .

[6]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[7]  Mohammad Amin Bassiri.  Interactional Feedback and the Impact of Attitude and Motivation on Noticing L2 Form , 2011 .

[8]  M. Engelmann.  The Philosophical Investigations , 2013 .

[9]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[10]  Jianfeng Gao,et al.  A Neural Network Approach to Context-Sensitive Generation of Conversational Responses , 2015, NAACL.

[11]  Wojciech Zaremba,et al.  Reinforcement Learning Neural Turing Machines - Revised , 2015 .

[12]  Jason Weston,et al.  Large-scale Simple Question Answering with Memory Networks , 2015, ArXiv.

[13]  Quoc V. Le,et al.  A Neural Conversational Model , 2015, ArXiv.

[14]  Wojciech Zaremba,et al.  Reinforcement Learning Neural Turing Machines , 2015, ArXiv.

[15]  Christopher D. Manning,et al.  Learning Language Games through Interaction , 2016, ACL.

[16]  David Vandyke,et al.  Continuously Learning Neural Dialogue Management , 2016, ArXiv.

[17]  Marc'Aurelio Ranzato,et al.  Sequence Level Training with Recurrent Neural Networks , 2015, ICLR.

[18]  Jason Weston,et al.  Dialog-based Language Learning , 2016, NIPS.

[19]  Jianfeng Gao,et al.  A Diversity-Promoting Objective Function for Neural Conversation Models , 2015, NAACL.

[20]  Jason Weston,et al.  Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks , 2015, ICLR.

[21]  Jason Weston,et al.  Key-Value Memory Networks for Directly Reading Documents , 2016, EMNLP.

[22]  Xiang Zhang,et al.  Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems , 2015, ICLR.

[23]  David Vandyke,et al.  A Network-based End-to-End Trainable Task-oriented Dialogue System , 2016, EACL.

[24]  Jason Weston,et al.  Learning End-to-End Goal-Oriented Dialog , 2016, ICLR.

[25]  Omer Levy,et al.  Published as a conference paper at ICLR 2018 S IMULATING A CTION D YNAMICS WITH N EURAL P ROCESS N ETWORKS , 2018 .