Exploring Variation of Natural Human Commands to a Robot in a Collaborative Navigation Task

Robot-directed communication is variable, and may change based on human perception of robot capabilities. To collect training data for a dialogue system and to investigate possible communication changes over time, we developed a Wizard-of-Oz study that (a) simulates a robot’s limited understanding, and (b) collects dialogues where human participants build a progressively better mental model of the robot’s understanding. With ten participants, we collected ten hours of human-robot dialogue. We analyzed the structure of instructions that participants gave to a remote robot before it responded. Our findings show a general initial preference for including metric information (e.g., move forward 3 feet) over landmarks (e.g., move to the desk) in motion commands, but this decreased over time, suggesting changes in perception.

[1]  Guido Bugmann,et al.  Corpus-Based Robotics: A Route Instruction Example , 2003 .

[2]  D. Harville.  Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems , 1977 .

[3]  A. Green,et al.  Applying the Wizard-of-Oz framework to cooperative service discovery and configuration , 2004, RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759).

[4]  Matthias Scheutz,et al.  Going Beyond Literal Command-Based Instructions: Extending Robotic Natural Language Interaction Capabilities , 2015, AAAI.

[5]  Rebecca J. Passonneau,et al.  Measuring Agreement on Set-valued Items (MASI) for Semantic and Pragmatic Annotation , 2006, LREC.

[6]  Kallirroi Georgila,et al.  SimSensei kiosk: a virtual human interviewer for healthcare decision support , 2014, AAMAS.

[7]  Changsong Liu,et al.  Collaborative Effort towards Common Ground in Situated Human-Robot Dialogue , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[8]  Anton Leuski,et al.  Radiobot-CFF: a spoken dialogue system for military training , 2006, INTERSPEECH.

[9]  Luke S. Zettlemoyer,et al.  Learning to Parse Natural Language Commands to a Robot Control System , 2012, ISER.

[10]  Oliver Lemon,et al.  Learning Effective Multimodal Dialogue Strategies from Wizard-of-Oz Data: Bootstrapping and Evaluation , 2008, ACL.

[11]  Matthias Scheutz,et al.  The Indiana “Cooperative Remote Search Task” (CReST) Corpus , 2010, LREC.

[12]  Gwyneth Doherty-Sneddon,et al.  The Reliability of a Dialogue Structure Coding Scheme , 1997, CL.

[13]  Matthew R. Walter,et al.  Learning models for following natural language directions in unknown environments , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Laurel D. Riek,et al.  Wizard of Oz studies in HRI , 2012, J. Hum. Robot Interact..

[15]  Alexander Koller,et al.  The GIVE-2 Corpus of Giving Instructions in Virtual Environments , 2010, LREC.

[16]  Jean Oh,et al.  Learning Qualitative Spatial Relations for Robotic Navigation , 2016, IJCAI.

[17]  Alexander I. Rudnicky,et al.  Comparing Spoken Language Route Instructions for Robots across Environment Representations , 2010, SIGDIAL Conference.

[18]  Anton Leuski,et al.  How Many Utterances Are Needed to Support Time-Offset Interaction? , 2015, FLAIRS Conference.

[19]  Dieter Fox,et al.  Following directions using statistical machine translation , 2010, 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[20]  A. W. Evans,et al.  Applying the Wizard-of-Oz Technique to Multimodal Human-Robot Dialogue , 2017, ArXiv.

[21]  Jayant Krishnamurthy,et al.  Jointly Learning to Parse and Perceive: Connecting Natural Language to the Physical World , 2013, TACL.

[22]  J. Guilford.  The Guilford‐Zimmerman Aptitude Survey , 1956 .

[23]  Kristin E. Schaefer,et al.  The Perception And Measurement Of Human-robot Trust , 2013 .

[24]  Luke S. Zettlemoyer,et al.  Weakly Supervised Learning of Semantic Parsers for Mapping Instructions to Actions , 2013, TACL.

[25]  Matthew R. Walter,et al.  Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation , 2011, AAAI.

[26]  David R. Traum,et al.  Creating spoken dialogue characters from corpora without annotations , 2007, INTERSPEECH.

[27]  Raymond J. Mooney,et al.  Learning to Interpret Natural Language Navigation Instructions from Observations , 2011, Proceedings of the AAAI Conference on Artificial Intelligence.

[28]  Eric Fosler-Lussier,et al.  SCARE: a Situated Corpus with Annotated Referring Expressions , 2008, LREC.

[29]  Benjamin Kuipers,et al.  Walk the Talk: Connecting Language, Knowledge, and Action in Route Instructions , 2006, AAAI.

[30]  M. Elsner,et al.  Giving Good Directions: Order of Mention Reflects Visual Salience , 2015, Front. Psychol..