Three-dimensional object reconstruction of symmetric objects by fusing visual and tactile sensing

In this work, we propose to reconstruct a complete three-dimensional (3-D) model of an unknown object by fusion of visual and tactile information while the object is grasped. Assuming the object is symmetric, a first hypothesis of its complete 3-D shape is generated. A grasp is executed on the object with a robotic manipulator equipped with tactile sensors. Given the detected contacts between the fingers and the object, the initial full object model including the symmetry parameters can be refined. This refined model will then allow the planning of more complex manipulation tasks. The main contribution of this work is an optimal estimation approach for the fusion of visual and tactile data applying the constraint of object symmetry. The fusion is formulated as a state estimation problem and solved with an iterated extended Kalman filter. The approach is validated experimentally using both artificial and real data from two different robotic platforms.

[1]  Nico Blodow,et al.  Combined 2D–3D categorization and classification for multimodal perception systems , 2011, Int. J. Robotics Res..

[2]  Jarmo Ilonen,et al.  Robust robot-camera calibration , 2011, 2011 15th International Conference on Advanced Robotics (ICAR).

[3]  Vijay Kumar,et al.  Robotic grasping and contact: a review , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[4]  Marc Toussaint,et al.  Gaussian process implicit surfaces for shape estimation and grasping , 2011, 2011 IEEE International Conference on Robotics and Automation.

[5]  Ling Xu,et al.  Physical Human Interactive Guidance: Identifying Grasping Principles From Human-Planned Grasps , 2012, IEEE Trans. Robotics.

[6]  Danijela Ristic-Durrant,et al.  Robust stereo-vision based 3D modelling of real-world objects for assistive robotic applications , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Yoshiki Uchikawa,et al.  A method of 3D object reconstruction by fusing vision with touch using internal models with global and local deformations , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[8]  Ashutosh Saxena,et al.  Efficient grasping from RGBD images: Learning using a new rectangle representation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[9]  Peter K. Allen,et al.  Integrating Vision and Touch for Object Recognition Tasks , 1988, Int. J. Robotics Res..

[10]  Helge J. Ritter,et al.  A Probabilistic Approach to Tactile Shape Reconstruction , 2011, IEEE Transactions on Robotics.

[11]  Gerd Hirzinger,et al.  Estimating finger contact location and object pose from contact measurements in 3D grasping , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[12]  Gabriel Taubin,et al.  Estimation of Planar Curves, Surfaces, and Nonplanar Space Curves Defined by Implicit Equations with Applications to Edge and Range Image Segmentation , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Dieter Fox,et al.  Sparse distance learning for object recognition combining RGB and depth information , 2011, 2011 IEEE International Conference on Robotics and Automation.

[14]  Takeo Kanade,et al.  Geometric reasoning for single image structure recovery , 2009, CVPR.

[15]  Frédéric Truchetet,et al.  Mesh Comparison Using Attribute Deviation Metric , 2004, Int. J. Image Graph..

[16]  John F. Canny,et al.  Planning optimal grasps , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[17]  J. Astola,et al.  Vector median filters , 1990, Proc. IEEE.

[18]  Danica Kragic,et al.  Mind the gap - robotic grasping under incomplete observation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[19]  Matei T. Ciocarlie,et al.  Contact-reactive grasping of objects with partial shape information , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Dieter Fox,et al.  RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments , 2012, Int. J. Robotics Res..

[21]  Jimmy A. Jørgensen,et al.  Grasping unknown objects using an Early Cognitive Vision system for general scene understanding , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Peter Allen.  Surface descriptions from vision and touch , 1984, ICRA.

[23]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[24]  Tamim Asfour,et al.  A potential field approach to dexterous tactile exploration of unknown objects , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[25]  Van-Duc Nguyen,et al.  Constructing Stable Grasps , 1989, Int. J. Robotics Res..

[26]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects using Vision , 2008, Int. J. Robotics Res..

[27]  Sebastian Thrun,et al.  Shape from symmetry , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[28]  Alberto Del Bimbo,et al.  Metric 3D reconstruction and texture acquisition of surfaces of revolution from a single uncalibrated view , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Siddhartha S. Srinivasa,et al.  Task Space Regions , 2011, Int. J. Robotics Res..

[30]  Markus Vincze,et al.  Shape-based depth image to 3D model matching and classification with inter-view similarity , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[31]  Joel W. Burdick,et al.  Fusion of stereo vision, force-torque, and joint sensors for estimation of in-hand object location , 2011, 2011 IEEE International Conference on Robotics and Automation.

[32]  汤晓鸥,et al.  Symmetric Piecewise Planar Object Reconstruction from a Single Image , 2011 .

[33]  Stefan Schaal,et al.  Probabilistic depth image registration incorporating nonvisual information , 2012, 2012 IEEE International Conference on Robotics and Automation.

[34]  Danica Kragic,et al.  Active 3D scene segmentation and detection of unknown objects , 2010, 2010 IEEE International Conference on Robotics and Automation.

[35]  Dieter Fox,et al.  Manipulator and object tracking for in-hand 3D object modeling , 2011, Int. J. Robotics Res..

[36]  Gary R. Bradski,et al.  Fast 3D recognition and pose using the Viewpoint Feature Histogram , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[37]  Dan Simon,et al.  Optimal State Estimation: Kalman, H∞, and Nonlinear Approaches , 2006 .