GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force

Tactile sensing is an important perception mode for robots, but the existing tactile technologies have multiple limitations. What kind of tactile information robots need, and how to use the information, remain open questions. We believe a soft sensor surface and high-resolution sensing of geometry should be important components of a competent tactile sensor. In this paper, we discuss the development of a vision-based optical tactile sensor, GelSight. Unlike the traditional tactile sensors which measure contact force, GelSight basically measures geometry, with very high spatial resolution. The sensor has a contact surface of soft elastomer, and it directly measures its deformation, both vertical and lateral, which corresponds to the exact object shape and the tension on the contact surface. The contact force, and slip can be inferred from the sensor’s deformation as well. Particularly, we focus on the hardware and software that support GelSight’s application on robot hands. This paper reviews the development of GelSight, with the emphasis in the sensing principle and sensor design. We introduce the design of the sensor’s optical system, the algorithm for shape, force and slip measurement, and the hardware designs and fabrication of different sensor versions. We also show the experimental evaluation on the GelSight’s performance on geometry and force measurement. With the high-resolution measurement of shape and contact force, the sensor has successfully assisted multiple robotic tasks, including material perception or recognition and in-hand localization for robot manipulation.

[1]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[2]  Thomas B. Sheridan,et al.  An optical tactile sensor for manipulators , 1984 .

[3]  Stefan Begej,et al.  Planar and finger-shaped optical tactile sensors for robotic applications , 1988, IEEE J. Robotics Autom..

[4]  Susan J. Lederman,et al.  Extracting object properties through haptic exploration. , 1993, Acta psychologica.

[5]  Roger W. Brockett,et al.  Reconstructing the Shape of a Deformable Membrane from Image Data , 2000, Int. J. Robotics Res..

[6]  Naoki Kawakami,et al.  Vision-based sensor for real-time measuring of surface traction fields , 2005, IEEE Computer Graphics and Applications.

[7]  V. Maheshwari,et al.  High-Resolution Thin-Film Device to Sense Texture by Touch , 2006, Science.

[8]  Veronica J. Santos,et al.  Biomimetic Tactile Sensor Array , 2008, Adv. Robotics.

[9]  I. Daubechies,et al.  Iteratively reweighted least squares minimization for sparse recovery , 2008, 0807.0575.

[10]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[11]  Jonathan Rossiter,et al.  Development of a tactile sensor based on biologically inspired edge encoding , 2009, 2009 International Conference on Advanced Robotics.

[12]  E. Adelson,et al.  Retrographic sensing for the measurement of surface texture and shape , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Giulio Sandini,et al.  Tactile Sensing—From Humans to Humanoids , 2010, IEEE Transactions on Robotics.

[14]  Naoki Kawakami,et al.  Finger-Shaped GelForce: Sensor for Measuring Surface Traction Fields for Robotic Hand , 2010, IEEE Transactions on Haptics.

[15]  W. B. Tiest.  Tactual perception of material properties , 2010, Vision Research.

[16]  Y. Ito,et al.  Robust Slippage Degree Estimation Based on Reference Update of Vision-Based Tactile Sensor , 2011, IEEE Sensors Journal.

[17]  Kaspar Althoefer,et al.  Tactile sensing for dexterous in-hand manipulation in robotics-A review , 2011 .

[18]  Edward H. Adelson,et al.  Microgeometry capture using an elastomeric sensor , 2011, ACM Trans. Graph..

[19]  Á. Tar,et al.  Development of a low cost 3D optical compliant tactile force sensor , 2011, 2011 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM).

[20]  Goro Obinata,et al.  Vision-Based Tactile Sensing and Shape Estimation Using a Fluid-Type Touchpad , 2012, IEEE Transactions on Automation Science and Engineering.

[21]  Edward H. Adelson,et al.  Sensing and Recognizing Surface Textures Using a GelSight Sensor , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  E. Adelson,et al.  Localization and Manipulation of Small Parts Using , 2014 .

[23]  Li.  Touching is Believing : Sensing and Analyzing Touch Information with GelSight by , 2015 .

[24]  Giorgio Metta,et al.  A new design of a fingertip for the iCub hand , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[25]  Edward H. Adelson,et al.  Measurement of shear and slip with a GelSight tactile sensor , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Gordon Cheng,et al.  Realizing whole-body tactile interactions with a self-organizing, multi-modal artificial skin on a humanoid robot , 2015, Adv. Robotics.

[27]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[28]  Véronique Perdereau,et al.  Tactile sensing in dexterous robot hands - Review , 2015, Robotics Auton. Syst..

[29]  Mark R. Cutkosky,et al.  Force and Tactile Sensing , 2016, Springer Handbook of Robotics, 2nd Ed..

[30]  Nathan F. Lepora,et al.  Tactile Manipulation With a TacThumb Integrated on the Open-Hand M2 Gripper , 2016, IEEE Robotics and Automation Letters.

[31]  Edward H. Adelson,et al.  Estimating object hardness with a GelSight touch sensor , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[32]  Edward H. Adelson,et al.  Connecting Look and Feel: Associating the Visual and Tactile Properties of Physical Materials , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Nathan F. Lepora,et al.  Addition of a Biomimetic Fingerprint on an Artificial Fingertip Enhances Tactile Spatial Acuity , 2017, IEEE Robotics and Automation Letters.

[34]  Edward H. Adelson,et al.  Improved GelSight tactile sensor for measuring geometry and slip , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[35]  Edward Adelson,et al.  Tracking objects with point clouds from vision and touch , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[36]  Nathan F. Lepora,et al.  Exploratory Tactile Servoing With Active Touch , 2017, IEEE Robotics and Automation Letters.

[37]  Andrew Owens,et al.  Shape-independent hardness estimation using deep learning and a GelSight tactile sensor , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).