Tactile Mapping and Localization from High-Resolution Tactile Imprints

This work studies the problem of shape reconstruction and object localization using a vision-based tactile sensor, GelSlim. The main contributions are the recovery of local shapes from contact, an approach to reconstruct the tactile shape of objects from tactile imprints, and an accurate method for object localization of previously reconstructed objects. The algorithms can be applied to a large variety of 3D objects and provide accurate tactile feedback for in-hand manipulation.Results show that by exploiting the dense tactile information we can reconstruct the shape of objects with high accuracy and do on-line object identification and localization, opening the door to reactive manipulation guided by tactile sensing. We provide videos and supplemental information in the project’s website web.mit.edu/mcube/research/tactile localization.html.

[1]  Ian Taylor,et al.  Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Frank Dellaert,et al.  Square Root SAM: Simultaneous Localization and Mapping via Square Root Information Smoothing , 2006, Int. J. Robotics Res..

[3]  Jakub W. Pachocki,et al.  Learning dexterous in-hand manipulation , 2018, Int. J. Robotics Res..

[4]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Jing Xiao,et al.  Shape-based object classification and recognition through continuum manipulation , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Edward H. Adelson,et al.  Localization and manipulation of small parts using GelSight tactile sensing , 2014, IROS.

[7]  Alberto Rodriguez,et al.  Dense Tactile Force Distribution Estimation using GelSlim and inverse FEM , 2018, ArXiv.

[8]  Kaspar Althoefer,et al.  iCLAP: shape recognition by combining proprioception and touch sensing , 2018, Autonomous Robots.

[9]  Andrew Owens,et al.  Shape-independent hardness estimation using deep learning and a GelSight tactile sensor , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[11]  Maria Bauzá,et al.  Tactile Regrasp: Grasp Adjustments via Simulated Tactile Transformations , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[12]  Jan Peters,et al.  Active tactile object exploration with Gaussian processes , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[13]  Gregory D. Hager,et al.  Object mapping, recognition, and localization from tactile geometry , 2011, 2011 IEEE International Conference on Robotics and Automation.

[14]  Mark Moll,et al.  Reconstructing the Shape and Motion of Unknown Objects with Active Tactile Sensors , 2002, WAFR.

[15]  Paul H. J. Kelly,et al.  SLAM++: Simultaneous Localisation and Mapping at the Level of Objects , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Giorgio Metta,et al.  Active contour following to explore object shape with robot touch , 2013, 2013 World Haptics Conference (WHC).

[17]  Edward H. Adelson,et al.  3D Shape Perception from Monocular Vision, Touch, and Shape Priors , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[18]  Jeannette Bohg,et al.  Three-dimensional object reconstruction of symmetric objects by fusing visual and tactile sensing , 2014, Int. J. Robotics Res..

[19]  Tamim Asfour,et al.  Local implicit surface estimation for haptic exploration , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[20]  Edward Adelson,et al.  Tracking objects with point clouds from vision and touch , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[21]  Stanley Finger,et al.  Cutaneous sensory spots and the “law of specific nerve energies”: history and development of ideas , 1999, Brain Research Bulletin.

[22]  Peter Englert,et al.  Active learning with query paths for tactile object shape exploration , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  Kuan-Ting Yu,et al.  Shape and pose recovery from planar pushing , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[24]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Achint Aggarwal,et al.  Object Recognition and Localization: The Role of Tactile Sensors , 2014, Sensors.

[26]  Helge J. Ritter,et al.  Using haptics to extract object shape from rotational manipulations , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  Edward H. Adelson,et al.  GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force , 2017, Sensors.

[28]  Jianhua Li,et al.  GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[29]  A. Landi.  Human Hand Function , 2007 .

[30]  Aude Billard,et al.  Bimanual compliant tactile exploration for grasping unknown objects , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[31]  Oussama Khatib,et al.  Global Localization of Objects via Touch , 2011, IEEE Transactions on Robotics.

[32]  Lorenzo Rosasco,et al.  Active perception: Building objects' models using tactile exploration , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[33]  Danica Kragic,et al.  Enhancing visual perception of shape through tactile glances , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[34]  Kaspar Althoefer,et al.  Localizing the object contact through matching tactile features with visual map , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[35]  Ravinder Dahiya,et al.  Robotic Tactile Perception of Object Properties: A Review , 2017, ArXiv.

[36]  Elliott Donlon,et al.  Maintaining Grasps within Slipping Bounds by Monitoring Incipient Slip , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[37]  Andrew T. Miller,et al.  Integration of Vision , Force and Tactile Sensing for Grasping , 1999 .

[38]  Siddhartha S. Srinivasa,et al.  Pose estimation for planar contact manipulation with manifold particle filters , 2015, Int. J. Robotics Res..

[39]  Pietro Falco,et al.  Cross-modal visuo-tactile object recognition using robotic active exploration , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[40]  Kostas Daniilidis,et al.  Active end-effector pose selection for tactile object recognition through Monte Carlo tree search , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).