Grasp recognition and mapping on humanoid robots

In this paper, we present a system for vision-based grasp recognition, mapping and execution on a humanoid robot to provide an intuitive and natural communication channel between humans and humanoids. This channel enables a human user to teach a robot how to grasp an object. The system comprises three components: human upper body motion capture system which provides the approaching direction towards an object, hand pose estimation and grasp recognition system, which provides the grasp type performed by the human as well as a grasp mapping and execution system for grasp reproduction on a humanoid robot with five-fingered hands. All three components are real-time and markerless. Once an object is reached, the hand posture is estimated, including hand orientation and grasp type. For the execution on a robot, hand posture and approach movement are mapped and optimized according to the kinematic limitations of the robot. Experimental results are performed on the humanoid robot ARMAR-IIIb.

[1]  Karl Rohr,et al.  Human Movement Analysis Based on Explicit Motion Models , 1997 .

[2]  M. Lopes,et al.  Motor Representations for Hand Gesture Recognition and Imitation , 2003 .

[3]  Tamim Asfour,et al.  Stereo-based 6D object localization for grasping with humanoid robot systems , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Stefan Schaal,et al.  Learning tasks from a single demonstration , 1997, Proceedings of International Conference on Robotics and Automation.

[5]  Mircea Nicolescu,et al.  A Review on Vision-Based Full DOF Hand Motion Estimation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[6]  ChangHwan Kim,et al.  Adaptation of human motion capture data to humanoid robots for motion imitation using optimization , 2006, Integr. Comput. Aided Eng..

[7]  Jitendra Malik,et al.  Tracking people with twists and exponential maps , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[8]  Olivier D. Faugeras,et al.  3D Articulated Models and Multiview Tracking with Physical Forces , 2001, Comput. Vis. Image Underst..

[9]  Katsushi Ikeuchi,et al.  Grasp Recognition Using The Contact Web , 1992, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Ales Ude,et al.  The Karlsruhe Humanoid Head , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[11]  Trevor Darrell,et al.  Fast pose estimation with parameter-sensitive hashing , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[12]  David J. Fleet,et al.  Model-based hand tracking with texture, shading and self-occlusions , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Alexandr Andoni,et al.  Near-Optimal Hashing Algorithms for Approximate Nearest Neighbor in High Dimensions , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[14]  William T. Freeman,et al.  Orientation Histograms for Hand Gesture Recognition , 1995 .

[15]  Tamim Asfour,et al.  ARMAR-III: An Integrated Humanoid Platform for Sensory-Motor Control , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[16]  N. Kamakura,et al.  Patterns of static prehension in normal hands. , 1980, The American journal of occupational therapy : official publication of the American Occupational Therapy Association.

[17]  Thomas S. Huang,et al.  Static Hand Gesture Recognition based on Local Orientation Histogram Feature Distribution Model , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[18]  Larry S. Davis,et al.  3-D model-based tracking of humans in action: a multi-view approach , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Chrystopher L. Nehaniv,et al.  Sensory-Motor Primitives as a Basis for Imitation: Linking Perception to Action and Biology to Robotics , 2002 .

[20]  Ales Ude,et al.  Programming full-body movements for humanoid robots by observation , 2004, Robotics Auton. Syst..

[21]  Stan Sclaroff,et al.  3D hand pose reconstruction using specialized mappings , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[22]  Björn Stenger,et al.  Filtering using a tree-based estimator , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[23]  Danica Kragic,et al.  Visual recognition of grasps for human-to-robot mapping , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Aude Billard,et al.  Discriminative and adaptive imitation in uni-manual and bi-manual tasks , 2006, Robotics Auton. Syst..

[25]  Tamim Asfour,et al.  Toward an Unified Representation for Imitation of Human Motion on Humanoids , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[26]  Ales Ude,et al.  Stereo-based Markerless Human Motion Capture for Humanoid Robot Systems , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[27]  Reinhard Koch,et al.  Monocular Body Pose Estimation by Color Histograms and Point Tracking , 2006, DAGM-Symposium.

[28]  Tamim Asfour,et al.  Robust real-time stereo-based markerless human motion capture , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[29]  Andrew Blake,et al.  Articulated body motion capture by annealed particle filtering , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[30]  Gordon Cheng,et al.  From Biologically Realistic Imitation to Robot Teaching Via Human Motor Learning , 2007, ICONIP.

[31]  Mark R. Cutkosky,et al.  On grasp choice, grasp models, and the design of hands for manufacturing tasks , 1989, IEEE Trans. Robotics Autom..

[32]  Rüdiger Dillmann,et al.  Visual servoing for humanoid grasping and manipulation tasks , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[33]  Zhaoqin Peng,et al.  Kinematics mapping and similarity evaluation of humanoid motion based on human motion capture , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[34]  Christopher G. Atkeson,et al.  Adapting human motion for the control of a humanoid robot , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[35]  Michael I. Mandel,et al.  Visual Hand Tracking Using Nonparametric Belief Propagation , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[36]  Tamim Asfour,et al.  Imitation of human motion on a humanoid robot using non-linear optimization , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.