Integrating 3D features and virtual visual servoing for hand-eye and humanoid robot pose estimation

In this paper, we propose an approach for vision-based pose estimation of a robot hand or full-body pose. The method is based on virtual visual servoing using a CAD model of the robot and it combines 2-D image features with depth features. The method can be applied to estimate either the pose of a robot hand or pose of the whole body given that its joint configuration is known. We present experimental results that show the performance of the approach as demonstrated on both a mobile humanoid robot and a stationary manipulator.

[1]  Mubarak Shah,et al.  A 3-dimensional sift descriptor and its application to action recognition , 2007, ACM Multimedia.

[2]  Danica Kragic,et al.  A strategy for grasping unknown objects based on co-planarity and colour information , 2010, Robotics Auton. Syst..

[3]  M. Vincze,et al.  BLORT-The Blocks World Robotic Vision Toolbox , 2010 .

[4]  Gary R. Bradski,et al.  Fast 3D recognition and pose using the Viewpoint Feature Histogram , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Thomas Feix,et al.  A comprehensive grasp taxonomy , 2009 .

[6]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[7]  Éric Marchand,et al.  Direct 3D servoing using dense depth maps , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Éric Marchand,et al.  Real-time markerless tracking for augmented reality: the virtual visual servoing framework , 2006, IEEE Transactions on Visualization and Computer Graphics.

[9]  É. Marchand,et al.  L'information mutuelle pour l'estimation visuelle directe de pose , 2012 .

[10]  Cordelia Schmid,et al.  An Affine Invariant Interest Point Detector , 2002, ECCV.

[11]  Xinhua Zhuang,et al.  Pose estimation from corresponding point data , 1989, IEEE Trans. Syst. Man Cybern..

[12]  Danica Kragic,et al.  Virtual Visual Servoing for Real-Time Robot Pose Estimation , 2011 .

[13]  Robert M. Haralick,et al.  Analysis and solutions of the three point perspective pose estimation problem , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Kostas Daniilidis,et al.  Linear Pose Estimation from Points or Lines , 2002, ECCV.

[15]  Nico Blodow,et al.  CAD-model recognition and 6DOF pose estimation using 3D cues , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[16]  Rüdiger Dillmann,et al.  Visual servoing for humanoid grasping and manipulation tasks , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[17]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[18]  Larry S. Davis,et al.  Iterative pose estimation using coplanar points , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Ronen Basri,et al.  Constructing implicit 3D shape models for pose estimation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[20]  Patrick Rives,et al.  Real-time Dense Visual Tracking under Large Lighting Variations , 2011, BMVC.

[21]  Danica Kragic,et al.  Dual arm manipulation - A survey , 2012, Robotics Auton. Syst..

[22]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[24]  Danica Kragic,et al.  Initialization and System Modeling in 3-D Pose Tracking , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[25]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Dieter Fox,et al.  A Scalable Tree-Based Approach for Joint Object and Pose Recognition , 2011, AAAI.

[27]  I. Du,et al.  Direct Methods , 1998 .