Interactive Natural Programming of Robots : Introductory Overview

Natural, multimodal interaction is a key issue of humanoid robots since a human-like appearance suggests human-like properties and behavior. Although human communication in all its facets seems to be hardly reachable today, interaction via verbal inand output or via gestures is a big step to meet this requirement. Besides these channels, recognition of the interaction context is important because e.g. human references are often incomplete and sloppy. From human behavior, several context types like advising, commenting and teaching can be derived. In this article, we present a system that supplies context dependant interaction with robots in these contexts. Interaction may be established with verbal and gestural channels. Furthermore, an interactive teaching technique called ”programming by demonstration” is presented for native and intuitive transferring human skills and behavior to human-like robots. Since manipulation skills from humans are very well developed but not easy to describe with standard programming methods, it is expected to ease robot programming significantly.

[1]  Richard Fikes,et al.  Learning and Executing Generalized Robot Plans , 1993, Artif. Intell..

[2]  M. Arbib.  Coordinated control programs for movements of the hand , 1985 .

[3]  Tom Michael Mitchell,et al.  Explanation-based generalization: A unifying view , 1986 .

[4]  Mark R. Cutkosky,et al.  On grasp choice, grasp models, and the design of hands for manufacturing tasks , 1989, IEEE Trans. Robotics Autom..

[5]  U. Rembold,et al.  KANTRA-human-machine interaction for intelligent robots using natural language , 1994, Proceedings of 1994 3rd IEEE International Workshop on Robot and Human Communication.

[6]  Masayuki Inaba,et al.  Learning by watching: extracting reusable task knowledge from visual observation of human performance , 1994, IEEE Trans. Robotics Autom..

[7]  Katsushi Ikeuchi,et al.  Toward automatic robot instruction from perception-temporal segmentation of tasks from human hand motion , 1993, IEEE Trans. Robotics Autom..

[8]  Larry S. Davis,et al.  Towards 3-D model-based tracking and recognition of human movement: a multi-view approach , 1995 .

[9]  Dorin Comaniciu,et al.  Robust analysis of feature spaces: color image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  José Santos-Victor,et al.  Robust visual tracking by an active observer , 1997, Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems. Innovative Robotics for Real-World Applications. IROS '97.

[11]  Kunikatsu Takase,et al.  Assembly motion teaching system using position/force simulator-generating control program , 1997, Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems. Innovative Robotics for Real-World Applications. IROS '97.

[12]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[13]  John Fry,et al.  Natural dialogue with the Jijo-2 office robot , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[14]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[15]  Holger Friedrich,et al.  Interaktive Programmierung von Manipulationssequenzen , 1998 .

[16]  Yoshiaki Shirai,et al.  Helping computer vision by verbal and nonverbal communication , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[17]  Rüdiger Dillmann,et al.  A sensor fusion approach for PbD , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[18]  Danica Kragic,et al.  A person following behaviour for a mobile robot , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[19]  R. Brooks,et al.  The cog project: building a humanoid robot , 1999 .

[20]  Pradeep K. Khosla,et al.  Gesture-based programming: a preliminary demonstration , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[21]  Jianwei Zhang,et al.  Interactive assembly by a two-arm robot agent , 1999, Robotics Auton. Syst..

[22]  Rüdiger Dillmann,et al.  Learning Robot Behaviour and Skills Based on Human Demonstration and Advice: The Machine Learning Paradigm , 2000 .

[23]  Rüdiger Dillmann,et al.  A general approach for modeling robots , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[24]  A. Steinhage,et al.  Learning by Doing : A Dynamic Architecture for Generating Adaptive Behavioral Sequences , 2000 .

[25]  Rüdiger Dillmann,et al.  A comparison of four fast vision based object recognition methods for programming by demonstration applications , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[26]  Tetsunori Kobayashi,et al.  A conversational robot utilizing facial and body expressions , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[27]  R. Dillmann,et al.  Sensor fusion approaches for observation of user actions in programming by demonstration , 2001, Conference Documentation International Conference on Multisensor Fusion and Integration for Intelligent Systems. MFI 2001 (Cat. No.01TH8590).

[28]  R. Dillmann,et al.  TEACHING SERVICE ROBOTS COMPLEX TASKS : PROGRAMMING BY DEMONSTRATION FOR WORKSHOP AND HOUSEHOLD ENVIRONMENTS , 2001 .

[29]  Magdalena D. Bugajska,et al.  Building a Multimodal Human-Robot Interface , 2001, IEEE Intell. Syst..

[30]  R. Zollner,et al.  Dynamic grasp recognition within the framework of programming by demonstration , 2001, Proceedings 10th IEEE International Workshop on Robot and Human Interactive Communication. ROMAN 2001 (Cat. No.01TH8591).

[31]  Atsuo Takanishi,et al.  Interactive biped locomotion based on visual/auditory information , 2002, Proceedings. 11th IEEE International Workshop on Robot and Human Interactive Communication.