Perception for mobile manipulation and grasping using active stereo

In this paper we present a comprehensive perception system with applications to mobile manipulation and grasping for personal robotics. Our approach makes use of dense 3D point cloud data acquired using stereo vision cameras by projecting textured light onto the scene. To create models suitable for grasping, we extract the supporting planes and model object clusters with different surface geometric primitives. The resultant decoupled primitive point clusters are then reconstructed as smooth triangular mesh surfaces, and their use is validated in grasping experiments using OpenRAVE [1]. To annotate the point cloud data with primitive geometric labels we make use of our previously proposed Fast Point Feature Histograms [2] and probabilistic graphical methods (Conditional Random Fields), and obtain a classification accuracy of 98.27% for different object geometries. We show the validity of our approach by analyzing the proposed system for the problem of building object models usable in grasping applications with the PR2 robot (see Figure 1).

[1]  Alberto Del Bimbo,et al.  Content-based retrieval of 3D models , 2006, TOMCCAP.

[2]  Gene H. Golub,et al.  Optimal Surface Smoothing as Filter Design , 1996, ECCV.

[3]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Kurt Konolige,et al.  Small Vision Systems: Hardware and Implementation , 1998 .

[5]  Alex Zelinsky,et al.  Learning OpenCV---Computer Vision with the OpenCV Library (Bradski, G.R. et al.; 2008)[On the Shelf] , 2009, IEEE Robotics & Automation Magazine.

[6]  Peter K. Allen,et al.  Graspit! A versatile simulator for robotic grasping , 2004, IEEE Robotics & Automation Magazine.

[7]  Michael Garland,et al.  Curvature maps for local shape comparison , 2005, International Conference on Shape Modeling and Applications 2005 (SMI' 05).

[8]  Remco C. Veltkamp,et al.  A survey of content based 3D shape retrieval methods , 2004, Proceedings Shape Modeling Applications, 2004..

[9]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[10]  James J. Kuffner,et al.  OpenRAVE: A Planning Architecture for Autonomous Robotics , 2008 .

[11]  Martial Hebert,et al.  Parts-based 3D object classification , 2004, CVPR 2004.

[12]  Chitra Dorai,et al.  3D object recognition: Representation and matching , 2000, Stat. Comput..

[13]  Peter Lindstrom,et al.  Out-of-core simplification of large polygonal models , 2000, SIGGRAPH.

[14]  Gilles Burel,et al.  Three-dimensional invariants and their application to object recognition , 1995, Signal Process..

[15]  Zoltan-Csaba Marton,et al.  On Fast Surface Reconstruction Methods for Large and Noisy Datasets , 2009, IEEE International Conference on Robotics and Automation.

[16]  Nico Blodow,et al.  Learning informative point classes for the acquisition of object model maps , 2008, 2008 10th International Conference on Control, Automation, Robotics and Vision.

[17]  W TangelderJohan,et al.  A survey of content based 3D shape retrieval methods , 2008 .

[18]  Craig Gotsman,et al.  Characterizing Shape Using Conformal Factors , 2008, 3DOR@Eurographics.

[19]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[20]  Mongi A. Abidi,et al.  Surface matching by 3D point's fingerprint , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.