Grasp observation and reproduction by humanoid robots using color camera and 3D sensor

In this paper, we present a system for grasp observation, mapping and execution on humanoid robots to provide intuitive and natural way of communication between humans and robots. This system enables a human user to teach a robot how to grasp an object. The system includes four main components: the hand movement tracking component which provides the approach direction toward the object, the pre-grasp hand pose estimation component which provides the grasp type performed by the human user, the contact points estimation component which provides the position on the object should be grasped, and the grasp mapping and execution component for grasp reproduction on humanoid robots with gripper hands.

[1]  Danica Kragic,et al.  Grasp recognition and mapping on humanoid robots , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[2]  Staffan Ekvall,et al.  Robot Task Learning from Human Demonstration , 2007 .

[3]  Tomás Lozano-Pérez,et al.  Imitation Learning of Whole-Body Grasps , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Oliver Kroemer,et al.  Learning robot grasping from 3-D images with Markov Random Fields , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Reinhard Klein,et al.  Efficient RANSAC for Point‐Cloud Shape Detection , 2007, Comput. Graph. Forum.

[6]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[7]  Danica Kragic,et al.  Visual recognition of grasps for human-to-robot mapping , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Tamio Arai,et al.  Teaching of grasp/graspless manipulation for industrial robots by human demonstration , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Thomas Feix,et al.  A comprehensive grasp taxonomy , 2009 .

[10]  P. E. Hotz,et al.  An adaptive learning mechanism for teaching a robotic hand to grasp , 2005 .

[11]  Satoru Hayamizu,et al.  Gesture recognition using HLAC features of PARCOR images and HMM based recognizer , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[12]  Takio Kurita,et al.  A New Scheme for Practical Flexible and Intelligent Vision Systems , 1988, MVA.

[13]  Stefano Caselli,et al.  Robot grasp synthesis from virtual demonstration and topology-preserving environment reconstruction , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.