How We Talk with Robots: Eliciting Minimally-Constrained Speech to Build Natural Language Interfaces and Capabilities

Industry, military, and academia are showing increasing interest in collaborative human-robot teaming in a variety of task contexts. Designing effective user interfaces for human-robot interaction is an ongoing challenge, and a variety of single and multiple-modality interfaces have been explored. Our work is to develop a bi-directional natural language interface for remote human-robot collaboration in physically situated tasks. When combined with a visual interface and audio cueing, we intend for the natural language interface to provide a naturalistic user experience that requires little training. Building the language portion of this interface requires first understanding how potential users would speak to the robot. In this paper, we describe our elicitation of minimally-constrained robot-directed language, observations about the users’ language behavior, and future directions for constructing an automated robotic system that can accommodate these language needs.

[1]  Stefanie Tellex,et al.  Toward understanding natural language directions , 2010, 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[2]  Saeid Nahavandi,et al.  Effects of realistic force feedback in a robotic assisted minimally invasive surgery system , 2014, Minimally invasive therapy & allied technologies : MITAT : official journal of the Society for Minimally Invasive Therapy.

[3]  Patrick Kenny,et al.  SimCoach: an intelligent virtual human system for providing healthcare information and support , 2011 .

[4]  Linda R. Elliott,et al.  Salience of Tactile Cues: An Examination of Tactor Actuator and Tactile Cue Characteristics , 2015 .

[5]  Jonathan Harris,et al.  Speech and gesture interfaces for squad-level human-robot teaming , 2014, Defense + Security Symposium.

[6]  Michael J. Barnes,et al.  Gesture-Based Controls for Robots: Overview and Implications for Use by Soldiers , 2016 .

[7]  Cory J. Hayes,et al.  Evaluating Robot Behavior in Response to Natural Language , 2018, HRI.

[8]  Joelle Pineau,et al.  Development and Validation of a Robust Speech Interface for Improved Human-Robot Interaction , 2009, Int. J. Soc. Robotics.

[9]  Kallirroi Georgila,et al.  New Dimensions in Testimony: Digitally Preserving a Holocaust Survivor's Interactive Storytelling , 2015, ICIDS.

[10]  Louis-Philippe Morency,et al.  MultiSense—Context-Aware Nonverbal Behavior Analysis Framework: A Psychological Distress Use Case , 2017, IEEE Transactions on Affective Computing.

[11]  Kallirroi Georgila,et al.  SimSensei kiosk: a virtual human interviewer for healthcare decision support , 2014, AAMAS.

[12]  Anton Leuski,et al.  Laying Down the Yellow Brick Road: Development of a Wizard-of-Oz Interface for Collecting Human-Robot Dialogue , 2017, ArXiv.

[13]  Alberto Poncela,et al.  Command-based voice teleoperation of a mobile robot via a human-robot interface , 2014, Robotica.

[14]  Eric Fosler-Lussier,et al.  SCARE: a Situated Corpus with Annotated Referring Expressions , 2008, LREC.

[15]  David R. Traum,et al.  Exploring the Implications of Virtual Human Research for Human-Robot Teams , 2015, HCI.

[16]  Keshav Bimbraw,et al.  Autonomous cars: Past, present and future a review of the developments in the last century, the present scenario and the expected future of autonomous vehicle technology , 2015, 2015 12th International Conference on Informatics in Control, Automation and Robotics (ICINCO).

[17]  Joseph J. LaViola,et al.  3D Gestural Interaction: The State of the Field , 2013 .

[18]  Bilge Mutlu,et al.  Modeling and Evaluating Narrative Gestures for Humanlike Robots , 2013, Robotics: Science and Systems.