Optical Tactile Sim-to-Real Policy Transfer via Real-to-Sim Tactile Image Translation

Simulation has recently become key for deep reinforcement learning to safely and efficiently acquire general and complex control policies from visual and proprioceptive inputs. Tactile information is not usually considered despite its direct relation to environment interaction. In this work, we present a suite of simulated environments tailored towards tactile robotics and reinforcement learning. A simple and fast method of simulating optical tactile sensors is provided, where high-resolution contact geometry is represented as depth images. Proximal Policy Optimisation (PPO) is used to learn successful policies across all considered tasks. A data-driven approach enables translation of the current state of a real tactile sensor to corresponding simulated depth images. This policy is implemented within a real-time control loop on a physical robot to demonstrate zeroshot sim-to-real policy transfer on several physically-interactive tasks requiring a sense of touch. Video results are available at https://sites.google.com/my.bristol.ac.uk/tactile-gym-sim2real/home. Code is available at https://github.com/ac-93/tactile gym.

[1]  Andrew J. Davison,et al.  Sim-to-Real Reinforcement Learning for Deformable Object Manipulation , 2018, CoRL.

[2]  Roberto Mart'in-Mart'in,et al.  robosuite: A Modular Simulation Framework and Benchmark for Robot Learning , 2020, ArXiv.

[3]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[4]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Jan Kautz,et al.  High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Raia Hadsell,et al.  From Pixels to Percepts: Highly Robust Edge Perception and Contour Following Using Deep Learning and an Optical Biomimetic Tactile Sensor , 2018, IEEE Robotics and Automation Letters.

[7]  Jonathan Rossiter,et al.  The TacTip Family: Soft Optical Tactile Sensors with 3D-Printed Biomimetic Morphologies , 2018, Soft robotics.

[8]  Mike Lambeta,et al.  TACTO: A Fast, Flexible and Open-source Simulator for High-Resolution Vision-based Tactile Sensors , 2020, ArXiv.

[9]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[10]  Bin Fang,et al.  Elastic Interaction of Particles for Robotic Tactile Simulation , 2020, ArXiv.

[11]  Yashraj S. Narang,et al.  Interpreting and Predicting Tactile Signals via a Physics-Based and Data-Driven Framework , 2020, Robotics: Science and Systems.

[12]  Raia Hadsell,et al.  Value constrained model-free continuous control , 2019, ArXiv.

[13]  Thomas Bi,et al.  Zero-Shot Sim-to-Real Transfer of Tactile Control Policies for Aggressive Swing-Up Manipulation , 2021, IEEE Robotics and Automation Letters.

[14]  E. Adelson,et al.  Retrographic sensing for the measurement of surface texture and shape , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[16]  Kristen Grauman,et al.  SoundSpaces: Audio-Visual Navigation in 3D Environments , 2020, ECCV.

[17]  Nathan F. Lepora,et al.  Optimal Deep Learning for Robot Touch: Training Accurate Pose Models of 3D Surfaces and Edges , 2020, IEEE Robotics & Automation Magazine.

[18]  Andrew J. Davison,et al.  RLBench: The Robot Learning Benchmark & Learning Environment , 2019, IEEE Robotics and Automation Letters.

[19]  Sergey Levine,et al.  Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Raffaello D'Andrea,et al.  Ground Truth Force Distribution for Learning-Based Tactile Sensing: A Finite Element Approach , 2019, IEEE Access.

[21]  N. Lepora,et al.  Goal-Driven Robotic Pushing Using Tactile and Proprioceptive Feedback , 2020, IEEE Transactions on Robotics.

[22]  Ahsan Habib,et al.  SkinSim: A simulation environment for multimodal robot skin , 2014, 2014 IEEE International Conference on Automation Science and Engineering (CASE).

[23]  Kazuhiro Shimonomura,et al.  Tactile Image Sensors Employing Camera: A Review , 2019, Sensors.

[24]  Mike Lambeta,et al.  DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor With Application to In-Hand Manipulation , 2020, IEEE Robotics and Automation Letters.

[25]  Po-wei Chou.  The Beta Policy for Continuous Control Reinforcement Learning , 2017 .

[26]  Frédo Durand,et al.  DiffTaichi: Differentiable Programming for Physical Simulation , 2020, ICLR.

[27]  Marcin Andrychowicz,et al.  Solving Rubik's Cube with a Robot Hand , 2019, ArXiv.

[28]  Paolo Paoletti,et al.  Generation of GelSight Tactile Images for Sim2Real Learning , 2021, IEEE Robotics and Automation Letters.

[29]  Raffaello D'Andrea,et al.  Design, Motivation and Evaluation of a Full-Resolution Optical Tactile Sensor , 2019, Sensors.

[30]  Pieter Abbeel,et al.  Reinforcement Learning with Augmented Data , 2020, NeurIPS.

[31]  Gaurav S. Sukhatme,et al.  NeuralSim: Augmenting Differentiable Simulators with Neural Networks , 2021, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[32]  Ilya Kostrikov,et al.  Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels , 2020, ArXiv.

[33]  Lorenz Wellhausen,et al.  Learning quadrupedal locomotion over challenging terrain , 2020, Science Robotics.

[34]  Russ Tedrake,et al.  Soft-bubble grippers for robust and perceptive manipulation , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[35]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[36]  Nathan F. Lepora,et al.  Pose-Based Servo Control with Soft Tactile Sensing , 2020, ArXiv.

[37]  Sergey Levine,et al.  How to train your robot with deep reinforcement learning: lessons we have learned , 2021, Int. J. Robotics Res..

[38]  Timo Korthals,et al.  Tactile Sensing and Deep Reinforcement Learning for In-Hand Manipulation Tasks , 2019 .

[39]  Mohi Khansari,et al.  RL-CycleGAN: Reinforcement Learning Aware Simulation-to-Real , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Sergey Levine,et al.  OmniTact: A Multi-Directional High-Resolution Touch Sensor , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[41]  Nathan F. Lepora,et al.  Sim-to-Real Transfer for Optical Tactile Sensing , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[42]  Henry Zhu,et al.  Soft Actor-Critic Algorithms and Applications , 2018, ArXiv.