Interpreting and predicting tactile signals for the SynTouch BioTac

In the human hand, high-density contact information provided by afferent neurons is essential for many human grasping and manipulation capabilities. In contrast, robotic tactile sensors, including the state-of-the-art SynTouch BioTac, are typically used to provide low-density contact information, such as contact location, center of pressure, and net force. Although useful, these data do not convey or leverage the rich information content that some tactile sensors naturally measure. This research extends robotic tactile sensing beyond reducedorder models through 1) the automated creation of a precise experimental tactile dataset for the BioTac over a diverse range of physical interactions, 2) a 3D finite element (FE) model of the BioTac, which complements the experimental dataset with high-density, distributed contact data, 3) neural-network-based mappings from raw BioTac signals to not only low-dimensional experimental data, but also high-density FE deformation fields, and 4) mappings from the FE deformation fields to the raw signals themselves. The high-density data streams can provide a far greater quantity of interpretable information for grasping and manipulation algorithms than previously accessible.

[1]  Veronica J. Santos,et al.  Biomimetic Tactile Sensor Array , 2008, Adv. Robotics.

[2]  Raffaello D'Andrea,et al.  Transfer learning for vision-based tactile sensing , 2018, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[3]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[4]  Fernando Torres,et al.  Generation of Tactile Data From 3D Vision and Target Robotic Grasps , 2020, IEEE Transactions on Haptics.

[5]  Jan Peters,et al.  Grip Stabilization of Novel Objects Using Slip Prediction , 2018, IEEE Transactions on Haptics.

[6]  M. T. Mason,et al.  Toward Robotic Manipulation , 2018, Annu. Rev. Control. Robotics Auton. Syst..

[7]  Gaurav S. Sukhatme,et al.  Force estimation and slip detection/classification for grip control using a biomimetic tactile sensor , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[8]  Jonathan Rossiter,et al.  The TacTip Family: Soft Optical Tactile Sensors with 3D-Printed Biomimetic Morphologies , 2018, Soft robotics.

[9]  Yanmei Li,et al.  A review of modeling of soft-contact fingers and stiffness control for dextrous manipulation in robotics , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[10]  Gerald E. Loeb,et al.  Haptic feature extraction from a biomimetic tactile sensor: Force, contact location and curvature , 2011, 2011 IEEE International Conference on Robotics and Biomimetics.

[11]  Raia Hadsell,et al.  From Pixels to Percepts: Highly Robust Edge Perception and Contour Following Using Deep Learning and an Optical Biomimetic Tactile Sensor , 2018, IEEE Robotics and Automation Letters.

[12]  Byron Boots,et al.  Robust Learning of Tactile Force Estimation through Robot Interaction , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[13]  Edward H. Adelson,et al.  GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force , 2017, Sensors.

[14]  Ravi Balasubramanian,et al.  The Human Hand as an Inspiration for Robot Hand Development , 2014, Springer Tracts in Advanced Robotics.

[15]  G.E. Loeb,et al.  Grip Control Using Biomimetic Tactile Sensing Systems , 2009, IEEE/ASME Transactions on Mechatronics.

[16]  Tucker Hermans,et al.  In-Hand Object-Dynamics Inference Using Tactile Fingertips , 2021, IEEE Transactions on Robotics.

[17]  Jeremy A. Marvel,et al.  Strategies for Improving and Evaluating Robot Registration Performance , 2018, IEEE Transactions on Automation Science and Engineering.

[18]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Raffaello D'Andrea,et al.  Ground Truth Force Distribution for Learning-Based Tactile Sensing: A Finite Element Approach , 2019, IEEE Access.

[20]  Edward H. Adelson,et al.  Active Clothing Material Perception Using Tactile Sensing and Deep Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[21]  Maria Bauza,et al.  Tactile Object Pose Estimation from the First Touch with Geometric Contact Rendering , 2020, CoRL.

[22]  Jeremy A. Marvel,et al.  Simplified framework for robot coordinate registration for manufacturing applications , 2016, 2016 IEEE International Symposium on Assembly and Manufacturing (ISAM).

[23]  Vlad Ciobanu,et al.  Point of Contact Location and Normal Force Estimation Using Biomimetical Tactile Sensors , 2014, 2014 Eighth International Conference on Complex, Intelligent and Software Intensive Systems.

[24]  Matei Ciocarlie,et al.  A Sensorized Multicurved Robot Finger With Data-Driven Touch Sensing via Overlapping Light Signals , 2020, IEEE/ASME Transactions on Mechatronics.

[25]  Yashraj S. Narang,et al.  Mechanically Versatile Soft Machines through Laminar Jamming , 2018 .

[26]  Giulio Sandini,et al.  Tactile Sensing—From Humans to Humanoids , 2010, IEEE Transactions on Robotics.

[27]  Mike Lambeta,et al.  DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor With Application to In-Hand Manipulation , 2020, IEEE Robotics and Automation Letters.

[28]  Russ Tedrake,et al.  Soft-bubble: A highly compliant dense geometry tactile sensor for robot manipulation , 2019, 2019 2nd IEEE International Conference on Soft Robotics (RoboSoft).

[29]  Chia-Hsien Lin,et al.  Estimating Point of Contact , Force and Torque in a Biomimetic Tactile Sensor with Deformable Skin , 2013 .

[30]  Jianwei Zhang,et al.  Simulation of the SynTouch BioTac Sensor , 2018, IAS.

[31]  Daniel Kappler,et al.  Riemannian Motion Policies , 2018, ArXiv.

[32]  Vijay Kumar,et al.  Robotic grasping and contact: a review , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[33]  Véronique Perdereau,et al.  Tactile sensing in dexterous robot hands - Review , 2015, Robotics Auton. Syst..

[34]  Matthew T. Mason,et al.  Mechanics of Robotic Manipulation , 2001 .

[35]  Kaspar Althoefer,et al.  Tactile sensing for dexterous in-hand manipulation in robotics-A review , 2011 .

[36]  Nathan Ida,et al.  Introduction to the Finite Element Method , 1997 .

[37]  Tomonori Yamamoto,et al.  Use of tactile feedback to control exploratory movements to characterize object compliance , 2012, Front. Neurorobot..

[38]  Yashraj S. Narang,et al.  Sim-to-Real for Robotic Tactile Sensing via Physics-Based Simulation and Learned Latent Projections , 2021, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[39]  Kevin Kelly,et al.  Comparative Peg-in-Hole Testing of a Force-Based Manipulation Controlled Robotic Hand , 2018, IEEE Transactions on Robotics.

[40]  Oliver Kroemer,et al.  A Review of Tactile Information: Perception and Action Through Touch , 2020, IEEE Transactions on Robotics.

[41]  Danica Kragic,et al.  Trends and challenges in robot manipulation , 2019, Science.

[42]  Jeremy A. Fishel,et al.  Signal processing and fabrication of a biomimetic tactile sensor array with thermal, force and microvibration modalities , 2009, 2009 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[43]  Ki-Suk Kim,et al.  Two-point discrimination values vary depending on test site, sex and test modality in the orofacial region: a preliminary study , 2017, Journal of applied oral science : revista FOB.

[44]  D. J. Montgomery,et al.  The physics of rubber elasticity , 1949 .

[45]  Elliott Donlon,et al.  Dense Tactile Force Estimation using GelSlim and inverse FEM , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[46]  G.E. Loeb,et al.  Deformable skin design to enhance response of a biomimetic tactile sensor , 2008, 2008 2nd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics.

[47]  Stefan Schaal,et al.  Surface tilt perception with a biomimetic tactile sensor , 2016, 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[48]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[49]  Christopher G. Atkeson,et al.  Recent progress in tactile sensing and sensors for robotic manipulation: can we turn tactile sensing into vision?1 , 2019, Adv. Robotics.

[50]  G. Loeb,et al.  Elastomeric Skin Selection for a Fluid-Filled Artificial Fingertip , 2013 .

[51]  Yashraj S. Narang,et al.  Interpreting and Predicting Tactile Signals via a Physics-Based and Data-Driven Framework , 2020, Robotics: Science and Systems.

[52]  Wei Chen,et al.  Tactile Sensors for Friction Estimation and Incipient Slip Detection—Toward Dexterous Robotic Manipulation: A Review , 2018, IEEE Sensors Journal.