Integrating Language, Vision and Action for Human Robot Dialog Systems

Developing a robot system that can interact directly with a human instructor in a natural way requires not only highly-skilled sensorimotor coordination and action planning on the part of the robot, but also the ability to understand and communicate with a human being in many modalities. A typical application of such a system is interactive assembly for construction tasks. A human communicator sharing a common view of the work area with the robot system instructs the latter by speaking to it in the same way that he would communicate with a human partner.

[1]  Candace L. Sidner,et al.  A First Experiment in Engagement for Human-Robot Interaction in Hosting Activities , 2005 .

[2]  J. Oberlander,et al.  Using Facial Feedback to Enhance Turn-Taking in a Multimodal Dialogue System , 2005 .

[3]  Ronnie W. Smith,et al.  Current and New Directions in Discourse and Dialogue , 2004 .

[4]  Jianwei Zhang,et al.  Instructing cooperating assembly robots through situated dialogues in natural language , 1997, Proceedings of International Conference on Robotics and Automation.

[5]  Tamim Asfour,et al.  A cognitive architecture for a humanoid robot: a first approach , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[6]  J. V. Kuppevelt,et al.  Advances in natural multimodal dialogue systems , 2005 .

[7]  H.-B. Kuntze,et al.  MULTI-SENSOR CONTROLLED SKILLS FOR HUMANOID ROBOTS , 2006 .

[8]  Estela Bicho,et al.  Goal-directed imitation for robots: A bio-inspired approach to action understanding and skill learning , 2006, Robotics Auton. Syst..

[9]  David Traum,et al.  The Information State Approach to Dialogue Management , 2003 .

[10]  Ann K. Syrdal,et al.  The AT&t German text-to-speech system: realistic linguistic description , 2002, INTERSPEECH.

[11]  Candace L. Sidner,et al.  Explorations in engagement for humans and robots , 2005, Artif. Intell..

[12]  David Salesin,et al.  Synthesizing realistic facial expressions from photographs , 1998, SIGGRAPH.

[13]  Alois Knoll,et al.  Human-Robot dialogue for joint construction tasks , 2006, ICMI '06.

[14]  Jianwei Zhang,et al.  A two-arm situated artificial communicator for human-robot cooperative assembly , 2003, IEEE Trans. Ind. Electron..

[15]  Michael White,et al.  Efficient Realization of Coordinate Structures in Combinatory Categorial Grammar , 2006 .

[16]  Pradeep K. Khosla,et al.  Manipulation task primitives for composing robot skills , 1997, Proceedings of International Conference on Robotics and Automation.

[17]  Ulrike Thomas,et al.  Error-tolerant execution of complex robot tasks based on skill primitives , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[18]  R. Moddemeijer.  On estimation of entropy and mutual information of continuous distributions , 1989 .

[19]  Andrea Lockerd Thomaz,et al.  Tutelage and Collaboration for Humanoid Robots , 2004, Int. J. Humanoid Robotics.

[20]  P. Kuehlein,et al.  Perspectives on dialogue in the new millennium , 2003 .

[21]  Alois Knoll,et al.  A Basic System for Multimodal Robot Instruction , 2003 .

[22]  Terrence Fong,et al.  The human-robot interaction operating system , 2006, HRI '06.

[23]  Michi Henning,et al.  A new approach to object-oriented middleware , 2004, IEEE Internet Computing.

[24]  Acm Press,et al.  Proceedings of the ... International Joint Conference on Autonomous Agents and Multiagent Systems , 2002 .

[25]  Alois Knoll,et al.  Distributed contract networks of sensor agents with adaptive reconfiguration: modelling, simulation, implementation and experiments , 2001, J. Frankl. Inst..

[26]  Alois Knoll,et al.  An Efficient and Robust Real-Time Contour Tracking System , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).

[27]  Martin Kay,et al.  Syntactic Process , 1979, ACL.

[28]  Xue Yan,et al.  iCat: an animated user-interface robot with personality , 2005, AAMAS '05.

[29]  Jianwei Zhang,et al.  Interactive assembly by a two-arm robot agent , 1999, Robotics Auton. Syst..

[30]  Alois Knoll,et al.  A two-arm situated artificial communicator for interactive assembly , 2001, Proceedings 10th IEEE International Workshop on Robot and Human Interactive Communication. ROMAN 2001 (Cat. No.01TH8591).