Perception and Grasping of Object Parts from Active Robot Exploration

Like humans, robots that need semantic perception and accurate estimation of the environment can increase their knowledge through active interaction with objects. This paper proposes a novel method for 3D object modeling for a robot manipulator with an eye-in-hand laser range sensor. Since the robot can only perceive the environment from a limited viewpoint, it actively manipulates a target object and generates a complete model by accumulation and registration of partial views. Three registration algorithms are investigated and compared in experiments performed in cluttered environments with complex rigid objects made of multiple parts. A data structure based on proximity graph, that encodes neighborhood relations in range scans, is also introduced to perform efficient range queries. The proposed method for 3D object modeling is applied to perform task-level manipulation. Indeed, once a complete model is available the object is segmented into its constituent parts and categorized. Object sub-parts that are relevant for the task and that afford a grasping action are identified and selected as candidate regions for grasp planning.

[1]  Dieter Fox,et al.  Autonomous generation of complete 3D object models using next best view manipulation planning , 2011, 2011 IEEE International Conference on Robotics and Automation.

[2]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Gabriel Zachmann,et al.  Point cloud surfaces using geometric proximity graphs , 2004, Comput. Graph..

[4]  Peter K. Allen,et al.  Grasp Planning via Decomposition Trees , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[5]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects using Vision , 2008, Int. J. Robotics Res..

[6]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[7]  Gerd Hirzinger,et al.  A surface-based Next-Best-View approach for automated 3D model completion of unknown objects , 2011, 2011 IEEE International Conference on Robotics and Automation.

[8]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[9]  Quoc V. Le,et al.  Grasping novel objects with depth segmentation , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Kamal K. Gupta,et al.  An autonomous six-DOF eye-in-hand system for in situ 3D object modeling , 2012, Int. J. Robotics Res..

[11]  Michael Beetz,et al.  Perception for mobile manipulation and grasping using active stereo , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[12]  Danica Kragic,et al.  Generalizing grasps across partly similar objects , 2012, 2012 IEEE International Conference on Robotics and Automation.

[13]  Rodney A. Brooks,et al.  Humanoid robots , 2002, CACM.

[14]  Markus Vincze,et al.  Efficient 3D Object Detection by Fitting Superquadrics to Range Image Data for Robot's Object Manipulation , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[15]  Stefano Caselli,et al.  Object categorization and grasping by parts from range scan data , 2012, 2012 IEEE International Conference on Robotics and Automation.

[16]  Roland Siegwart,et al.  Object classification based on a geometric grammar with a range camera , 2009, 2009 IEEE International Conference on Robotics and Automation.

[17]  Stefano Caselli,et al.  Part-based robot grasp planning from human demonstration , 2011, 2011 IEEE International Conference on Robotics and Automation.

[18]  Zoltan-Csaba Marton,et al.  Probabilistic categorization of kitchen objects in table settings with a composite sensor , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Gary M. Bone,et al.  Automated modeling and robotic grasping of unknown three-dimensional objects , 2008, 2008 IEEE International Conference on Robotics and Automation.

[20]  Ramakant Nevatia,et al.  Part-Based 3D Descriptions of Complex Objects from a Single Image , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Shengyong Chen,et al.  Vision sensor planning for 3-D model acquisition , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[22]  Faouzi Ghorbel,et al.  A simple and efficient approach for 3D mesh approximate convex decomposition , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[23]  Ioannis Stamos,et al.  Automated model acquisition from range images with view planning , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Christian Schlegel,et al.  Mobile Manipulation in Service Robotics: Scene and Object Recognition with Manipulator-Mounted Laser Ranger , 2010, ISR/ROBOTIK.

[25]  Larry H. Matthies,et al.  End-to-end dexterous manipulation with deliberate interactive estimation , 2012, 2012 IEEE International Conference on Robotics and Automation.

[26]  Martin D. Levine,et al.  Representing 3-D Objects in Range Images Using Geons , 1996, Comput. Vis. Image Underst..

[27]  Stefan Schaal,et al.  Learning to grasp under uncertainty , 2011, 2011 IEEE International Conference on Robotics and Automation.

[28]  Rüdiger Dillmann,et al.  Towards shape-based visual object categorization for humanoid robots , 2011, 2011 IEEE International Conference on Robotics and Automation.

[29]  Masayuki Inaba,et al.  On-line next best grasp selection for in-hand object 3D modeling with dual-arm coordination , 2012, 2012 IEEE International Conference on Robotics and Automation.

[30]  Luis Enrique Sucar,et al.  View planning for 3D object reconstruction , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[31]  Robert B. Fisher,et al.  Class-based Recognition of 3D Objects Represented by Volumetric Primitives , 1996, BMVC.

[32]  Danica Kragic,et al.  Birth of the Object: Detection of Objectness and Extraction of Object Shape through Object-Action complexes , 2008, Int. J. Humanoid Robotics.

[33]  Richard Pito,et al.  A Solution to the Next Best View Problem for Automated Surface Acquisition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Matei T. Ciocarlie,et al.  Data-driven grasping with partial sensor data , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[35]  Quoc V. Le,et al.  High-accuracy 3D sensing for mobile manipulation: Improving object detection and door opening , 2009, 2009 IEEE International Conference on Robotics and Automation.

[36]  Scott P. Johnson,et al.  Systems in development: motor skill acquisition facilitates three-dimensional object completion. , 2010, Developmental psychology.

[37]  I. Biederman Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[38]  Anis Sahbani,et al.  A hybrid approach for grasping 3D objects , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[39]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[40]  Bir Bhanu,et al.  THREE-POINT SEED METHOD FOR THE EXTRACTION OF PLANAR FACES FROM RANGE DATA. , 1982 .

[41]  Mohammed Bennamoun,et al.  Three-Dimensional Model-Based Object Recognition and Segmentation in Cluttered Scenes , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Pierre Payeur,et al.  Scan-based registration of range measurements , 2002, IMTC/2002. Proceedings of the 19th IEEE Instrumentation and Measurement Technology Conference (IEEE Cat. No.00CH37276).

[43]  Roland Siegwart,et al.  Incremental object part detection toward object classification in a sequence of noisy range images , 2008, 2008 IEEE International Conference on Robotics and Automation.

[44]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[45]  Martin Buss,et al.  Realtime segmentation of range data using continuous nearest neighbors , 2009, 2009 IEEE International Conference on Robotics and Automation.

[46]  Nico Blodow,et al.  Close-range scene segmentation and reconstruction of 3D point cloud maps for mobile manipulation in domestic environments , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[47]  Alberto Del Bimbo,et al.  3D Mesh decomposition using Reeb graphs , 2009, Image Vis. Comput..

[48]  Stefan Funke,et al.  How much geometry it takes to reconstruct a 2-manifold in R3 , 2010, JEAL.

[49]  Min-Hyung Choi,et al.  An Improved ICP Algorithm Based on the Sensor Projection for Automatic 3D Registration , 2004, MICAI.

[50]  Danica Kragic,et al.  Minimum volume bounding box decomposition for shape approximation in robot grasping , 2008, 2008 IEEE International Conference on Robotics and Automation.

[51]  Dejan Pangercic,et al.  Real-time CAD model matching for mobile manipulation and grasping , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[52]  Robert B. Fisher,et al.  Model-driven grouping and recognition of generic object parts from single images , 1997, Robotics Auton. Syst..

[53]  Dieter Fox,et al.  Manipulator and object tracking for in-hand 3D object modeling , 2011, Int. J. Robotics Res..

[54]  Stefano Caselli,et al.  A 3D shape segmentation approach for robot grasping by parts , 2012, Robotics Auton. Syst..

[55]  Sunghee Choi,et al.  The power crust , 2001, SMA '01.

[56]  Justus H. Piater,et al.  Development of Object and Grasping Knowledge by Robot Exploration , 2010, IEEE Transactions on Autonomous Mental Development.

[57]  Alireza Bab-Hadiashar,et al.  Range image segmentation using surface selection criterion , 2006, IEEE Transactions on Image Processing.

[58]  Aaron Hertzmann,et al.  Learning 3D mesh segmentation and labeling , 2010, SIGGRAPH 2010.

[59]  Dmitry Berenson,et al.  Grasp planning in complex scenes , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[60]  Roland Siegwart,et al.  Segmentation and Unsupervised Part-based Discovery of Repetitive Objects , 2010, Robotics: Science and Systems.

[61]  Michael Zillich,et al.  Grasping unknown objects based on 2½D range data , 2008, 2008 IEEE International Conference on Automation Science and Engineering.

[62]  E. Reed The Ecological Approach to Visual Perception , 1989 .