DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality

Recent work has demonstrated the ability of deep reinforcement learning (RL) algorithms to learn complex robotic behaviours in simulation, including in the domain of multi-fingered manipulation. However, such models can be challenging to transfer to the real world due to the gap between simulation and reality. In this paper, we present our techniques to train a) a policy that can perform robust dexterous manipulation on an anthropomorphic robot hand and b) a robust pose estimator suitable for providing reliable real-time information on the state of the object being manipulated. Our policies are trained to adapt to a wide range of conditions in simulation. Consequently, our vision-based policies significantly outperform the best vision policies in the literature on the same reorientation task and are competitive with policies that are given privileged state information via motion capture systems. Our work reaffirms the possibilities of sim-to-real transfer for dexterous manipulation in diverse kinds of hardware and simulator setups, and in our case, with the Allegro Hand and Isaac Gym GPU-based simulation. Furthermore, it opens up possibilities for researchers to achieve such results with commonly-available, affordable robot hands and cameras. Videos of the resulting policy and supplementary information, including experiments and demos, can be found at https://dextreme.org/ .

[1]  Vikash Kumar,et al.  MyoSuite: A Contact-rich Simulation Suite for Musculoskeletal Motor Control , 2022, L4DC.

[2]  Prafulla Dhariwal,et al.  Hierarchical Text-Conditional Image Generation with CLIP Latents , 2022, ArXiv.

[3]  B. Bäuml,et al.  Learning Purely Tactile In-Hand Manipulation with a Torque-Controlled Hand , 2022, 2022 International Conference on Robotics and Automation (ICRA).

[4]  Kostas E. Bekris,et al.  Complex In-Hand Manipulation via Compliance-Enabled Finger Gaiting and Multi-Modal Planning , 2022, IEEE Robotics and Automation Letters.

[5]  Ankur Handa,et al.  Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger , 2021, 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Pieter Abbeel,et al.  Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning , 2021, ArXiv.

[7]  Pulkit Agrawal,et al.  A System for General In-Hand Object Re-Orientation , 2021, CoRL.

[8]  Miles Macklin,et al.  Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning , 2021, NeurIPS Datasets and Benchmarks.

[9]  Romain Br'egier,et al.  Deep Regression on Manifolds: A 3D Rotation Case Study , 2021, 2021 International Conference on 3D Vision (3DV).

[10]  Alec Radford,et al.  Zero-Shot Text-to-Image Generation , 2021, ICML.

[11]  Joonho Lee,et al.  Circus ANYmal: A Quadruped Learning Dexterous Manipulation with Its Limbs , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Lorenz Wellhausen,et al.  Learning quadrupedal locomotion over challenging terrain , 2020, Science Robotics.

[13]  Wojciech Zaremba,et al.  Predicting Sim-to-Real Transfer with Probabilistic Dynamics Models , 2020, ArXiv.

[14]  Jaime Fern'andez del R'io,et al.  Array programming with NumPy , 2020, Nature.

[15]  Mark Chen,et al.  Language Models are Few-Shot Learners , 2020, NeurIPS.

[16]  Wan-Yen Lo,et al.  Accelerating 3D deep learning with PyTorch3D , 2019, SIGGRAPH Asia 2020 Courses.

[17]  Jakub W. Pachocki,et al.  Learning dexterous in-hand manipulation , 2018, Int. J. Robotics Res..

[18]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[19]  Marcin Andrychowicz,et al.  Solving Rubik's Cube with a Robot Hand , 2019, ArXiv.

[20]  Seong Joon Oh,et al.  CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[21]  Joonho Lee,et al.  Learning agile and dynamic motor skills for legged robots , 2019, Science Robotics.

[22]  Masatoshi Ishikawa,et al.  Rubik's Cube Handling Using a High-Speed Multi-Fingered Hand and a High-Speed Vision System , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  Dieter Fox,et al.  Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects , 2018, CoRL.

[24]  Marcin Andrychowicz,et al.  Asymmetric Actor Critic for Image-Based Robot Learning , 2017, Robotics: Science and Systems.

[25]  Marcin Andrychowicz,et al.  Sim-to-Real Transfer of Robotic Control with Dynamics Randomization , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Sergey Levine,et al.  Learning Dexterous Manipulation Policies from Experience and Imitation , 2016, ArXiv.

[27]  Sergey Levine,et al.  Optimal control with learned local models: Application to dexterous manipulation , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[28]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[29]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[31]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[32]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[33]  Inman Harvey,et al.  Noise and the Reality Gap: The Use of Simulation in Evolutionary Robotics , 1995, ECAL.

[34]  S. Shankar Sastry,et al.  Grasping and Coordinated Manipulation by a Multifingered Robot Hand , 1989, Int. J. Robotics Res..

[35]  S. Gruber,et al.  Robot hands and the mechanics of manipulation , 1987, Proceedings of the IEEE.

[36]  John J. Craig,et al.  Articulated hands: Force control and kinematic issues , 1981 .