Balancing Efficiency and Coverage in Human-Robot Dialogue Collection

We describe a multi-phased Wizard-of-Oz approach to collecting human-robot dialogue in a collaborative search and navigation task. The data is being used to train an initial automated robot dialogue system to support collaborative exploration tasks. In the first phase, a wizard freely typed robot utterances to human participants. For the second phase, this data was used to design a GUI that includes buttons for the most common communications, and templates for communications with varying parameters. Comparison of the data gathered in these phases show that the GUI enabled a faster pace of dialogue while still maintaining high coverage of suitable responses, enabling more efficient targeted data collection, and improvements in natural language understanding using GUI-collected data. As a promising first step towards interactive learning, this work shows that our approach enables the collection of useful training data for navigation-based HRI tasks.

[1]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[2]  Guido Bugmann,et al.  Corpus-Based Robotics: A Route Instruction Example , 2003 .

[3]  Stefanie Tellex,et al.  Clarifying commands with information-theoretic human-robot dialog , 2013, HRI 2013.

[4]  David R. Traum,et al.  Dialogue Structure Annotation for Multi-Floor Interaction , 2018, LREC.

[5]  Matthias Scheutz,et al.  Going Beyond Literal Command-Based Instructions: Extending Robotic Natural Language Interaction Capabilities , 2015, AAAI.

[6]  David R. Traum,et al.  Assessing Agreement in Human-Robot Dialogue Strategies: A Tale of Two Wizards , 2016, IVA.

[7]  Andrea Lockerd Thomaz,et al.  Effects of nonverbal communication on efficiency and robustness in human-robot teamwork , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Alexander I. Rudnicky,et al.  Miscommunication Recovery in Physically Situated Dialogue , 2015, SIGDIAL Conference.

[9]  Changsong Liu,et al.  Collaborative Language Grounding Toward Situated Human-Robot Dialogue , 2017, AI Mag..

[10]  Nikolaos Mavridis,et al.  A review of verbal and non-verbal human-robot interactive communication , 2014, Robotics Auton. Syst..

[11]  Susan L. Epstein,et al.  Embedded Wizardry , 2011, SIGDIAL Conference.

[12]  Amy Isard,et al.  Evaluating Description and Reference Strategies in a Cooperative Human-Robot Dialogue System , 2009, IJCAI.

[13]  Laurel D. Riek,et al.  Wizard of Oz studies in HRI , 2012, J. Hum. Robot Interact..

[14]  Anton Leuski,et al.  NPCEditor: Creating Virtual Human Dialogue Using Information Retrieval Techniques , 2011, AI Mag..

[15]  Stephanie Rosenthal,et al.  Dynamic generation and refinement of robot verbalization , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[16]  Nigel Gilbert,et al.  Simulating speech systems , 1991 .

[17]  Alexander I. Rudnicky,et al.  Exploring Spoken Dialog Interaction in Human-Robot Teams , 2009 .

[18]  D. Harville.  Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems , 1977 .

[19]  Eric Horvitz,et al.  Directions robot: in-the-wild experiences and lessons learned , 2014, AAMAS.

[20]  Matthew R. Walter,et al.  Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation , 2011, AAAI.

[21]  Alexander I. Rudnicky,et al.  The TeamTalk Corpus: Route Instructions in Open Spaces , 2011 .

[23]  Ross A. Knepper,et al.  Recovering from failure by asking for help , 2015, Auton. Robots.

[24]  Albert Rizzo,et al.  SimCoach: An Online Intelligent Virtual Agent System for Breaking Down Barriers to Care for Service Members and Veterans , 2013 .

[25]  Michael F. Schober,et al.  Spatial Dialogue between Partners with Mismatched Abilities , 2009, Spatial Language and Dialogue.

[26]  Matthew R. Walter,et al.  Learning models for following natural language directions in unknown environments , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[27]  Joëlle Coutaz,et al.  Applying the Wizard of Oz Technique to the Study of Multimodal Systems , 1993, EWHCI.

[28]  Kallirroi Georgila,et al.  SimSensei kiosk: a virtual human interviewer for healthcare decision support , 2014, AAMAS.

[29]  Boyang Li,et al.  Learning and Reusing Dialog for Repeated Interactions with a Situated Social Agent , 2017, IVA.

[30]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[31]  Manfred Tscheligi,et al.  User experience evaluation with a Wizard of Oz approach: Technical and methodological considerations , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[32]  Jeffrey Nichols,et al.  Chorus: a crowd-powered conversational assistant , 2013, UIST.

[33]  Pierre Lison,et al.  Situated Dialogue Processing for Human-Robot Interaction , 2010, Cognitive Systems.

[34]  Wayne S. Zimmerman,et al.  The Guilford-Zimmerman Aptitude Survey. , 1948 .

[35]  Matthias Scheutz,et al.  An Overview of the Distributed Integrated Cognition Affect and Reflection DIARC Architecture , 2018, Intelligent Systems, Control and Automation: Science and Engineering.

[36]  A. W. Evans,et al.  Applying the Wizard-of-Oz Technique to Multimodal Human-Robot Dialogue , 2017, ArXiv.

[37]  J. Guilford.  The Guilford‐Zimmerman Aptitude Survey , 1956 .

[38]  David R. Traum,et al.  Exploring Variation of Natural Human Commands to a Robot in a Collaborative Navigation Task , 2017, RoboNLP@ACL.

[39]  Benjamin Kuipers,et al.  Walk the Talk: Connecting Language, Knowledge, and Action in Route Instructions , 2006, AAAI.