A multisensor system for observation of user actions in programming by demonstration

Good observation of a manipulation presentation performed by a human teacher is crucial to further processing steps in programming by demonstration in interactive robot programming. This paper outlines a concept of how this can be done using visual and finger measuring sensors. The input sources include: a data glove which classifies several gestures and grasps, an active stereo vision, and a fixed ceiling camera. The hardware used is presented together with the technical concepts of processing and the acquired sensor information fusion, so called elementary cognitive operators. All the sensor sources use time-efficient algorithms, since sensor data must be processed in real-time.

[1]  Ales Ude,et al.  Trajectory generation from noisy positions of object features for teaching robot paths , 1993, Robotics Auton. Syst..

[2]  Mark R. Cutkosky,et al.  On grasp choice, grasp models, and the design of hands for manufacturing tasks , 1989, IEEE Trans. Robotics Autom..

[3]  R. Dillmann,et al.  An efficient image processing algorithm for high-level skill acquisition , 1999, Proceedings of the 1999 IEEE International Symposium on Assembly and Task Planning (ISATP'99) (Cat. No.99TH8470).

[4]  Masayuki Inaba,et al.  Learning by watching: extracting reusable task knowledge from visual observation of human performance , 1994, IEEE Trans. Robotics Autom..

[5]  Jochen Triesch,et al.  Robust classification of hand postures against complex backgrounds , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[6]  Jochen Triesch,et al.  GripSee: A Gesture-Controlled Robot for Object Perception and Manipulation , 1999, Auton. Robots.

[7]  Dean Pomerleau,et al.  Efficient Training of Artificial Neural Networks for Autonomous Navigation , 1991, Neural Computation.

[8]  Rüdiger Dillmann,et al.  A sensor fusion approach for PbD , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[9]  Michael Isard,et al.  A Smoothing Filter for CONDENSATION , 1998, ECCV.

[10]  Peter Weckesser,et al.  Accuracy of scene reconstruction with an active stereo vision system using motorized zoom lenses , 1994, Other Conferences.

[11]  Sing Bing Kang,et al.  Robot instruction by human demonstration , 1995 .

[12]  Jochen Triesch,et al.  Robotic Gesture Recognition , 1997, Gesture Workshop.

[13]  Rüdiger Dillmann,et al.  3D-icon based user interaction for robot programming by demonstration , 1997, Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA'97. 'Towards New Computational Principles for Robotics and Automation'.