Are You Talking to Me? Dialogue Systems Supporting Mixed Teams of Humans and Robots

This position paper describes an approach to building spoken dialogue systems for environments containing multiple human speakers and hearers, and multiple robotic speakers and hearers. We address the issue, for robotic hearers, of whether the speech they hear is intended for them, or more likely to be intended for some other hearer. We will describe data collected during a series of experiments involving teams of multiple human and robots (and other software participants), and some preliminary results for distinguishing robot-directed speech from human-directed speech. The domain of these experiments is Mars-analogue planetary exploration. These Mars-analogue field studies involve two subjects in simulated planetary space suits doing geological exploration with the help of 1-2 robots, supporting software agents, a habitat communicator and links to a remote science team. The two subjects are performing a task (geological exploration) which requires them to speak with each other while also speaking with their assistants. The technique used here is to use a probabilistic context-free grammar language model in the speech recognizer that is trained on prior robot-directed speech. Intuitively, the recognizer will give higher confidence to an utterance if it is similar to utterances that have been directed to the robot in the past.

[1]  Beth Ann Hockey,et al.  Simple Kernel Methods for Identification of Cross-Talk and Misrecognition , .

[2]  A. W. Evans,et al.  A Scale MOUT Facility for Studying Human-Robot Interaction and Control , 2004 .

[3]  Douglas E. Appelt,et al.  GEMINI: A Natural Language System for Spoken-Language Understanding , 1993, ACL.

[4]  Victor Zue,et al.  GALAXY-II: a reference architecture for conversational system development , 1998, ICSLP.

[5]  Karsten L. Worm A Model for Robust Processing of Spontaneous Speech by Integrating Viable Fragments , 1998, COLING-ACL.

[6]  Oliver Lemon,et al.  Combining Acoustic and Pragmatic Features to Predict Recognition Performance in Spoken Dialogue Systems , 2004, ACL.

[7]  Rieks op den Akker,et al.  Towards Automatic Addressee Identification in Multi-party Dialogues , 2004, SIGDIAL Workshop.

[8]  David L. Akin,et al.  The Effects of Extravehicular Activity Gloves on Human Hand Performance , 2001 .

[9]  William J. Clancey Roles for agent assistants in field science: understanding personal projects and collaboration , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[10]  David L. Akin,et al.  Development and Testing of a Space Suit Analogue for Neutral Buoyancy EVA Research , 2002 .

[11]  Adam Cheyer,et al.  CommandTalk: A Spoken-Language Interface for Battlefield Simulations , 1997, ANLP.

[12]  Beth Ann Hockey,et al.  Practical Issues in Compiling Typed Unification Grammars for Speech Recognition , 2001, ACL.

[13]  Mark-Jan Nederhof,et al.  Robust grammatical analysis for spoken dialogue systems , 1999, Natural Language Engineering.

[14]  Maarten Sierhuis,et al.  Intelligence for Human-Assistant Planetary Surface Robots , 2006 .

[15]  James F. Allen,et al.  Towards Conversational Human-Computer Interaction , 2000 .

[16]  W. Clancey,et al.  Brahms: A Multiagent Modeling Environment for simulating social phenomenon , 2001 .

[17]  Alexander I. Rudnicky,et al.  Expanding the Scope of the ATIS Task: The ATIS-3 Corpus , 1994, HLT.

[18]  Beth Ann Hockey,et al.  A Compact Architecture for Dialogue Management Based on Scripts and Meta-Outputs , 2000, ANLP.

[19]  Douglas E. Appelt,et al.  A Template Matcher for Robust NL Interpretation , 1991, HLT.

[20]  Eric Horvitz,et al.  Continuous listening for unconstrained spoken dialog , 2000, INTERSPEECH.

[21]  Maarten Sierhuis,et al.  Automating CapCom Using Mobile Agents and Robotic Assistants , 2005 .

[22]  Illah R. Nourbakhsh,et al.  Human-robot teaming for search and rescue , 2005, IEEE Pervasive Computing.

[23]  James F. Allen,et al.  Toward Conversational Human-Computer Interaction , 2001, AI Mag..

[24]  Candace L. Sidner,et al.  Where to look: a study of human-robot engagement , 2004, IUI '04.