Hybrid chat and task dialogue for more engaging HRI using reinforcement learning

Most of today's task-based spoken dialogue systems perform poorly if the user goal is not within the system's task domain. On the other hand, chatbots cannot perform tasks involving robot actions but are able to deal with unforeseen user input. To overcome the limitations of each of these separate approaches and be able to exploit their strengths, we present and evaluate a fully autonomous robotic system using a novel combination of task-based and chat-style dialogue in order to enhance the user experience with human-robot dialogue systems. We employ Reinforcement Learning (RL) to create a scalable and extensible approach to combining chat and task-based dialogue for multimodal systems. In an evaluation with real users, the combined system was rated as significantly more “pleasant” and better met the users' expectations in a hybrid task+chat condition, compared to the task-only condition, without suffering any significant loss in task completion.

[1]  Marilyn A. Walker,et al.  PARADISE: A Framework for Evaluating Spoken Dialogue Agents , 1997, ACL.

[2]  Stefanie Tellex,et al.  Toward understanding natural language directions , 2010, HRI 2010.

[3]  Martin Schrepp,et al.  Construction and Evaluation of a User Experience Questionnaire , 2008, USAB.

[4]  Alexander I. Rudnicky,et al.  Learning Conversational Systems that Interleave Task and Non-Task Content , 2017, IJCAI.

[5]  Shimei Pan,et al.  Designing and Evaluating an Adaptive Spoken Dialogue System , 2002, User Modeling and User-Adapted Interaction.

[6]  Oliver Lemon,et al.  Combining Chat and Task-Based Multimodal Dialogue for More Engaging HRI: A Scalable Method Using Reinforcement Learning , 2017, HRI.

[7]  Natàlia Hurtós,et al.  ROSPlan: Planning in the Robot Operating System , 2015, ICAPS.

[8]  Aswin van Woudenberg.  A Chatbot Dialogue Manager - Chatbots and Dialogue Systems: A Hybrid Approach , 2014 .

[9]  Milica Gasic,et al.  POMDP-Based Statistical Spoken Dialog Systems: A Review , 2013, Proceedings of the IEEE.

[10]  Fabrice Lefèvre,et al.  Reinforcement-learning based dialogue system for human-robot interactions with socially-inspired rewards , 2015, Comput. Speech Lang..

[11]  Oliver Lemon,et al.  Strategic Dialogue Management via Deep Reinforcement Learning , 2015, NIPS 2015.

[12]  Raymond J. Mooney,et al.  Learning to Interpret Natural Language Navigation Instructions from Observations , 2011, Proceedings of the AAAI Conference on Artificial Intelligence.

[13]  Wolfgang Wahlster,et al.  Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics , 1997 .

[14]  Michael F. McTear,et al.  Book Review: Spoken Dialogue Technology: Toward the Conversational User Interface, by Michael F. McTear , 2002, CL.

[15]  Oliver Lemon,et al.  Reinforcement Learning for Adaptive Dialogue Systems - ReadingSample , 2017 .

[16]  Pedro U. Lima,et al.  Petri Net Plans , 2011, Autonomous Agents and Multi-Agent Systems.

[17]  Alexiei Dingli,et al.  Building a Hybrid: Chatterbot - Dialog System , 2013, TSD.

[18]  Dana Kulic,et al.  Measurement Instruments for the Anthropomorphism, Animacy, Likeability, Perceived Intelligence, and Perceived Safety of Robots , 2009, Int. J. Soc. Robotics.