Motion capture from demonstrator's viewpoint and its application to robot teaching

In this paper, we propose a kind of "teaching by demonstration" method, aiming at its application to humanoid robots at home in the future. The demonstrator's motion is captured by a pair of stereo cameras mounted on his/her head, locating very close to his/her eyes. By tracking the landmarks attached to the demonstrator's hand and the working environment, one can estimate not only the demonstrator's hand motion but also his/her head motion, which can be used for the active vision system. Experimental result shows the effectiveness of the proposed framework.

[1]  Daniel E. Koditschek,et al.  An active visual estimator for dexterous manipulation , 1996, IEEE Trans. Robotics Autom..

[2]  Tsuneo Yoshikawa,et al.  Accurate image overlay on video see-through HMDs using vision and accelerometers , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[3]  Takuya Kondo,et al.  Incremental tracking of human actions from multiple views , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[4]  Greg Welch,et al.  SCAAT: incremental tracking with incomplete information , 1997, SIGGRAPH.

[5]  Donald B. Gennery,et al.  Visual tracking of known three-dimensional objects , 1992, International Journal of Computer Vision.

[6]  Takeo Kanade,et al.  Vision-Based Object Registration for Real-Time Image Overlay , 1995, CVRMed.

[7]  Kevin Nickels,et al.  Model-based tracking of complex articulated objects , 2001, IEEE Trans. Robotics Autom..

[8]  Ronald Azuma,et al.  Hybrid inertial and vision tracking for augmented reality registration , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[9]  Masayuki Inaba,et al.  Learning by watching: extracting reusable task knowledge from visual observation of human performance , 1994, IEEE Trans. Robotics Autom..

[10]  Katsushi Ikeuchi,et al.  Toward an assembly plan from observation. I. Task recognition with polyhedral objects , 1994, IEEE Trans. Robotics Autom..

[11]  Fumio Miyazaki,et al.  Manipulator Control by Visual Ser-voing with the Stereo Vision , 1993 .

[12]  Mark A. Livingston,et al.  Superior augmented reality registration by integrating landmark tracking and magnetic tracking , 1996, SIGGRAPH.