Why and How to Model Multi-Modal Interaction for a Mobile Robot Companion

Verbal and non-verbal interaction capabilities for robots are often studied isolated from each other in current research trend because they largely contribute to different aspects of interaction. For a robot companion that needs to be both useful and social, however, these capabilities have to be considered in a unified, complex interaction context. In this paper we present two case studies in such a context that clearly reveal the strengths and limitations of these modalities and advocate their complementary benefits for human-robot interaction. Motivated by this evidence we propose a powerful interaction framework which addresses common features of interactional and propositional information instead of their differences, as popular in many other works in this field, and models them using one single principle: grounding.

[1]  Staffan Larsson,et al.  Information state and dialogue management in the TRINDI dialogue move engine toolkit , 2000, Natural Language Engineering.

[2]  J. Cassell,et al.  More Than Just Another Pretty Face: Embodied Conversational Interface Agents , 1999 .

[3]  Gerhard Sagerer,et al.  A computational model of multi-modal grounding , 2006, ACL 2006.

[4]  David R. Traum,et al.  Embodied agents for multi-party dialogue in immersive virtual worlds , 2002, AAMAS '02.

[5]  Rodney A. Brooks,et al.  A robot that walks; emergent behaviors from a carefully evolved network , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[6]  Aude Billard,et al.  Challenges in designing the body and the mind of an interactive robot , 2005 .

[7]  Kristinn R. Thórisson.  Gandalf: an embodied humanoid capable of real-time multimodal dialogue with people , 1997, AGENTS '97.

[8]  Janet E. Cahn,et al.  A Psychological Model of Grounding and Repair in Dialog , 1999 .

[9]  Cynthia Breazeal,et al.  Designing sociable robots , 2002 .

[10]  Tetsuo Ono,et al.  Development and evaluation of an interactive humanoid robot "Robovie" , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[11]  H. H. Clark.  Arenas of language use , 1993 .

[12]  Stanley Peters,et al.  Information states in a multimodal dialogue system for human-robot conversation , 2001 .

[13]  Sebastian Lang,et al.  Providing the basis for human-robot-interaction: a multi-modal attention system for a mobile robot , 2003, ICMI '03.

[14]  J. Cassell,et al.  Embodied conversational agents , 2000 .

[15]  Gerhard Sagerer,et al.  A dialog system for comparative user studies on robot verbal behavior , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[16]  Kerstin Dautenhahn,et al.  What is a robot companion - friend, assistant or butler? , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  M. C. Caselli,et al.  Gesturing in mother-child interactions * , 1999 .

[18]  W. Levelt,et al.  Speaking: From Intention to Articulation , 1990 .

[19]  Takio Kurita,et al.  Integrated Natural Spoken Dialogue System of Jijo-2 Mobile Robot for Office Services , 1999, AAAI/IAAI.

[20]  Justine Cassell,et al.  Human conversation as a system framework: designing embodied conversational agents , 2001 .

[21]  Jannik Fritsch,et al.  Human-style interaction with a robot for cooperative learning of scene objects , 2005, ICMI '05.

[22]  David R Traum,et al.  Towards a Computational Theory of Grounding in Natural Language Conversation , 1991 .