RGB-D object pose estimation in unstructured environments

We present an object pose estimation approach exploiting both geometric depth and photometric color information available from an RGB-D sensor. In contrast to various efforts relying on object segmentation with a known background structure, our approach does not depend on the segmentation and thus exhibits superior performance in unstructured environments. Inspired by a voting-based approach employing an oriented point pair feature, we present a voting-based approach which further incorporates color information from the RGB-D sensor and which exploits parallel power of the modern parallel computing architecture. The proposed approach is extensively evaluated with three state-of-the-art approaches on both synthetic and real datasets, and our approach outperforms the other approaches in terms of both computation time and accuracy. A point pair feature describing both geometric shape and photometric color and its application to a voting-based 6-DOF object pose estimation.Parallel algorithms significantly accelerating the voting process on modern GPU architectures.Effective in unstructured scenes in which the prevailing segmentation-based approaches may not be applicable.

[1]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Lawrence G. Roberts,et al.  Machine Perception of Three-Dimensional Solids , 1963, Outstanding Dissertations in the Computer Sciences.

[3]  Darius Burschka,et al.  Rigid 3D geometry matching for grasping of known objects in cluttered scenes , 2012, Int. J. Robotics Res..

[4]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Björn Stenger,et al.  A new distance for scale-invariant 3D shape recognition and registration , 2011, 2011 International Conference on Computer Vision.

[6]  Ming-Yu Liu,et al.  Voting-based pose estimation for robotic assembly using a 3D sensor , 2012, 2012 IEEE International Conference on Robotics and Automation.

[7]  Chitra Dorai,et al.  COSMOS - A Representation Scheme for 3D Free-Form Objects , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Federico Tombari,et al.  A combined texture-shape descriptor for enhanced 3D feature matching , 2011, 2011 18th IEEE International Conference on Image Processing.

[9]  Paul H. J. Kelly,et al.  SLAM++: Simultaneous Localisation and Mapping at the Level of Objects , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Ramesh Raskar,et al.  Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging , 2004 .

[11]  Clark F. Olson,et al.  Automatic target recognition by matching oriented edge pixels , 1997, IEEE Trans. Image Process..

[12]  Rüdiger Dillmann,et al.  The KIT object models database: An object model database for object recognition, localization and manipulation in service robotics , 2012, Int. J. Robotics Res..

[13]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[14]  Vincent Lepetit,et al.  Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes , 2012, ACCV.

[15]  Björn Stenger,et al.  Demisting the Hough Transform for 3D Shape Recognition and Registration , 2014, International Journal of Computer Vision.

[16]  Henrik I. Christensen,et al.  Robust 3D visual tracking using particle filtering on the special Euclidean group: A combined approach of keypoint and edge features , 2011, 2011 IEEE International Conference on Robotics and Automation.

[17]  Juan José Pantrigo,et al.  Particle filter on GPUs for real-time tracking , 2004, SIGGRAPH '04.

[18]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[19]  Gérard G. Medioni,et al.  Structural Indexing: Efficient 3-D Object Recognition , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Hanspeter Pfister,et al.  Fast and automatic object pose estimation for range images on the GPU , 2009, Machine Vision and Applications.

[21]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[22]  Alex Zelinsky,et al.  Learning OpenCV---Computer Vision with the OpenCV Library (Bradski, G.R. et al.; 2008)[On the Shelf] , 2009, IEEE Robotics & Automation Magazine.

[23]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[24]  Tamim Asfour,et al.  6-DoF model-based tracking of arbitrarily shaped 3D objects , 2011, 2011 IEEE International Conference on Robotics and Automation.

[25]  Ramesh Raskar,et al.  Vision-guided Robot System for Picking Objects by Casting Shadows , 2010, Int. J. Robotics Res..

[26]  Mohammed Bennamoun,et al.  3D Object Recognition in Cluttered Scenes with Local Surface Features: A Survey , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Mohammed Bennamoun,et al.  Automatic Correspondence for 3d Modeling: an Extensive Review , 2005, Int. J. Shape Model..

[28]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Markus Vincze,et al.  Multimodal cue integration through Hypotheses Verification for RGB-D object recognition and 6DOF pose estimation , 2013, 2013 IEEE International Conference on Robotics and Automation.

[30]  Eric Wahl,et al.  Surflet-pair-relation histograms: a statistical 3D-shape representation for rapid classification , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[31]  Dieter Fox,et al.  A Scalable Tree-Based Approach for Joint Object and Pose Recognition , 2011, AAAI.

[32]  Henrik I. Christensen,et al.  RGB-D object tracking: A particle filter approach on GPU , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Jitendra Malik,et al.  Recognizing Objects in Range Data Using Regional Point Descriptors , 2004, ECCV.

[34]  Nassir Navab,et al.  Model globally, match locally: Efficient and robust 3D object recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[36]  Simon Winkelbach,et al.  Efficient bin-picking and grasp planning based on depth data , 2013, 2013 IEEE International Conference on Robotics and Automation.

[37]  Gary R. Bradski,et al.  Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library , 2016 .

[38]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[39]  Rama Chellappa,et al.  Pose estimation in heavy clutter using a multi-flash camera , 2010, 2010 IEEE International Conference on Robotics and Automation.

[40]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[41]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[42]  Pieter Abbeel,et al.  Multimodal blending for high-accuracy instance recognition , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[43]  Andreas Birk,et al.  Utilizing color information in 3D scan-registration using planar-patches matching , 2012, 2012 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI).

[44]  Henrik I. Christensen,et al.  3D pose estimation of daily objects using an RGB-D camera , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[45]  Henrik I. Christensen,et al.  3D textureless object detection and tracking: An edge-based approach , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[46]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[47]  Robert C. Bolles,et al.  Parametric Correspondence and Chamfer Matching: Two New Techniques for Image Matching , 1977, IJCAI.

[48]  Rama Chellappa,et al.  Fast directional chamfer matching , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[49]  Jun Li,et al.  Mobile bin picking with an anthropomorphic service robot , 2013, 2013 IEEE International Conference on Robotics and Automation.

[50]  Siddhartha S. Srinivasa,et al.  Object recognition and full pose registration from a single image for robotic manipulation , 2009, 2009 IEEE International Conference on Robotics and Automation.

[51]  Federico Tombari,et al.  Unique Signatures of Histograms for Local Surface Description , 2010, ECCV.

[52]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[53]  Kazuhiro Otsuka,et al.  Real-time Visual Tracker by Stream Processing , 2009, J. Signal Process. Syst..

[54]  Peter Meer,et al.  Simultaneous multiple 3D motion estimation via mode finding on Lie groups , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[55]  Martial Hebert,et al.  Rapid object indexing using locality sensitive hashing and joint 3D-signature space estimation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Nico Blodow,et al.  CAD-model recognition and 6DOF pose estimation using 3D cues , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[57]  Gérard G. Medioni,et al.  3D object recognition in range images using visibility context , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[58]  Mark Fiala,et al.  ARTag, a fiducial marker system using digital techniques , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[59]  Aly A. Farag,et al.  CSIFT: A SIFT Descriptor with Color Invariant Characteristics , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[60]  Mohammed Bennamoun,et al.  Three-Dimensional Model-Based Object Recognition and Segmentation in Cluttered Scenes , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[62]  Ramesh Raskar,et al.  Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging , 2004, SIGGRAPH 2004.

[63]  FuaPascal,et al.  Gradient Response Maps for Real-Time Detection of Textureless Objects , 2012 .

[64]  Gary R. Bradski,et al.  Fast 3D recognition and pose using the Viewpoint Feature Histogram , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[65]  Vicente Arévalo,et al.  A compact planar-patch descriptor based on color , 2014, 2014 11th International Conference on Informatics in Control, Automation and Robotics (ICINCO).