Dependable multimodal communication and interaction with robotic assistants

To advance research in the field of multimodal human-robot communication we designed and built the humanoid robot Hermes. Equipped with an omnidirectional undercarriage and two manipulator arms it combines visual, kinesthetic, tactile, and auditory sensing with natural spoken language input and output and body expressions for natural communication and interaction with humans. Hermes was successfully tested in an extended six-month experiment in a museum where only naive users interacted with the robot. They chatted with Hermes in several languages and requested various services. Multimodal communication and an understanding of the current situation by the robot turned out to be the key to success.

[1]  Jianwei Zhang,et al.  Instructing cooperating assembly robots through situated dialogues in natural language , 1997, Proceedings of International Conference on Robotics and Automation.

[2]  Mark C. Torrance,et al.  Natural communication with robots , 1994 .

[3]  Brian Scassellati,et al.  How to build robots that make friends and influence people , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[4]  Stanley Peters,et al.  Information states in a multimodal dialogue system for human-robot conversation , 2001 .

[5]  R. Bischoff,et al.  Integrating vision, touch and natural language in the control of a situation-oriented behavior-based humanoid robot , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[6]  Futoshi Asano,et al.  Map Learning of an Office Conversant Mobile Robot, Jijo-2, by Dialogue-Guided Navigation , 1998 .

[7]  Wolfram Burgard,et al.  Probabilistic Algorithms and the Interactive Museum Tour-Guide Robot Minerva , 2000, Int. J. Robotics Res..

[8]  Wolfgang Wahlster,et al.  KANTRA - A Natural Language Interface for Intelligent Robots , 2003 .