Tactile SLAM: Real-time inference of shape and pose from planar pushing

Tactile perception is central to robot manipulation in unstructured environments. However, it requires contact, and a mature implementation must infer object models while also accounting for the motion induced by the interaction. In this work, we present a method to estimate both object shape and pose in real-time from a stream of tactile measurements. This is applied towards tactile exploration of an unknown object by planar pushing. We consider this as an online SLAM problem with a nonparametric shape representation. Our formulation of tactile inference alternates between Gaussian process implicit surface regression and pose estimation on a factor graph. Through a combination of local Gaussian processes and fixed-lag smoothing, we infer object shape and pose in real-time. We evaluate our system across different objects in both simulated and real-world planar pushing tasks.

[1]  Ruzena Bajcsy,et al.  Recovery of Parametric Models from Range Images: The Case for Superquadrics with Global Deformations , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Andrew Fitzgibbon,et al.  Gaussian Process Implicit Surfaces , 2006 .

[3]  Siddhartha S. Srinivasa,et al.  Pose estimation for contact manipulation with manifold particle filters , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Oussama Khatib,et al.  Global Localization of Objects via Touch , 2011, IEEE Transactions on Robotics.

[5]  Edward H. Adelson,et al.  GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force , 2017, Sensors.

[6]  Helge J. Ritter,et al.  A Probabilistic Approach to Tactile Shape Reconstruction , 2011, IEEE Transactions on Robotics.

[7]  Frank Dellaert,et al.  Factor Graphs for Robot Perception , 2017, Found. Trends Robotics.

[8]  Jianhua Li,et al.  GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[9]  Jonghyuk Kim,et al.  Continuous occupancy maps using overlapping local Gaussian processes , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Alberto Rodriguez,et al.  Feedback Control of the Pusher-Slider System: A Story of Hybrid and Underactuated Contact Dynamics , 2016, WAFR.

[11]  Michael Kaess,et al.  MH-iSAM2: Multi-hypothesis iSAM using Bayes Tree and Hypo-tree , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[12]  Ravinder Dahiya,et al.  Robotic Tactile Perception of Object Properties: A Review , 2017, ArXiv.

[13]  Kuan-Ting Yu,et al.  Realtime State Estimation with Tactile and Visual Sensing. Application to Planar Manipulation , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Edward Adelson,et al.  Tracking objects with point clouds from vision and touch , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[15]  Siddhartha S. Srinivasa,et al.  A Planning Framework for Non-Prehensile Manipulation under Clutter and Uncertainty , 2012, Autonomous Robots.

[16]  Johannes A. Stork,et al.  Ensemble of Sparse Gaussian Process Experts for Implicit Surface Mapping with Streaming Data , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Anil K. Jain,et al.  A modified Hausdorff distance for object matching , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[18]  Matthew T. Mason,et al.  Mechanics and Planning of Manipulator Pushing Operations , 1986 .

[19]  Jeffrey C. Trinkle,et al.  A dynamic Bayesian approach to real-time estimation and filtering in grasp acquisition , 2013, 2013 IEEE International Conference on Robotics and Automation.

[20]  Chad DeChant,et al.  Shape completion enabled robotic grasping , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  Dieter Fox,et al.  DART: dense articulated real-time tracking with consumer depth cameras , 2015, Auton. Robots.

[22]  Joel W. Burdick,et al.  Fusion of stereo vision, force-torque, and joint sensors for estimation of in-hand object location , 2011, 2011 IEEE International Conference on Robotics and Automation.

[23]  Marc Toussaint,et al.  Gaussian process implicit surfaces for shape estimation and grasping , 2011, 2011 IEEE International Conference on Robotics and Automation.

[24]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[25]  Kazuo Tanie,et al.  Manipulation And Active Sensing By Pushing Using Tactile Feedback , 1992, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Byron Boots,et al.  Joint Inference of Kinematic and Force Trajectories with Visuo-Tactile Sensing , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[27]  James F. Blinn,et al.  A Generalization of Algebraic Surface Drawing , 1982, TOGS.

[28]  Soo-Hong Lee,et al.  Fixture planning with friction , 1991 .

[29]  Mark Moll,et al.  Reconstructing the Shape and Motion of Unknown Objects with Active Tactile Sensors , 2002, WAFR.

[30]  Giorgio Metta,et al.  Active contour following to explore object shape with robot touch , 2013, 2013 World Haptics Conference (WHC).

[31]  R L Klatzky,et al.  Identifying objects by touch: An “expert system” , 1985, Perception & psychophysics.

[32]  Kuan-Ting Yu,et al.  Shape and pose recovery from planar pushing , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[33]  Hugh F. Durrant-Whyte,et al.  Simultaneous localization and mapping: part I , 2006, IEEE Robotics & Automation Magazine.

[34]  Maria Bauza,et al.  Tactile Mapping and Localization from High-Resolution Tactile Imprints , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[35]  Danica Kragic,et al.  Dexterous grasping under shape uncertainty , 2016, Robotics Auton. Syst..

[36]  A. Ruina,et al.  Planar sliding with dry friction Part 1. Limit surface and moment function , 1991 .

[37]  Marc Toussaint,et al.  Active Multi-Contact Continuous Tactile Exploration with Gaussian Process Differential Entropy , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[38]  Sudeep Sarkar,et al.  Multi-scale superquadric fitting for efficient shape and pose recovery of unknown objects , 2013, 2013 IEEE International Conference on Robotics and Automation.

[39]  Siddhartha S. Srinivasa,et al.  Pose estimation for planar contact manipulation with manifold particle filters , 2015, Int. J. Robotics Res..

[40]  Kuan-Ting Yu,et al.  More than a million ways to be pushed. A high-fidelity experimental dataset of planar pushing , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[41]  Francesco Zanichelli,et al.  Efficient exploration and recognition of convex objects based on haptic perception , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[42]  Frank Dellaert,et al.  iSAM2: Incremental smoothing and mapping using the Bayes tree , 2012, Int. J. Robotics Res..

[43]  Helge J. Ritter,et al.  Correcting pose estimates during tactile exploration of object shape: a neuro-robotic study , 2014, 4th International Conference on Development and Learning and on Epigenetic Robotics.

[44]  Daniel D. Lee,et al.  Online Continuous Mapping using Gaussian Process Implicit Surfaces , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[45]  Marc Toussaint,et al.  Uncertainty aware grasping and tactile exploration , 2013, 2013 IEEE International Conference on Robotics and Automation.