Real2Sim2Real Transfer for Control of Cable-driven Robots via a Differentiable Physics Engine

Tensegrity robots, composed of rigid rods and flexible cables, exhibit high strength-to-weight ratios and significant deformations, which enable them to navigate unstructured terrains and survive harsh impacts. They are hard to control, however, due to high dimensionality, complex dynamics, and a coupled architecture. Physics-based simulation is a promising avenue for developing locomotion policies that can be transferred to real robots. Nevertheless, modeling tensegrity robots is a complex task due to a substantial sim2real gap. To address this issue, this paper describes a Real2Sim2Real (R2S2R) strategy for tensegrity robots. This strategy is based on a differentiable physics engine that can be trained given limited data from a real robot. These data include offline measurements of physical properties, such as mass and geometry for various robot components, and the observation of a trajectory using a random control policy. With the data from the real robot, the engine can be iteratively refined and used to discover locomotion policies that are directly transferable to the real robot. Beyond the R2S2R pipeline, key contributions of this work include computing non-zero gradients at contact points, a loss function for matching tensegrity locomotion gaits, and a trajectory segmentation technique that avoids conflicts in gradient evaluation during training. Multiple iterations of the R2S2R process are demonstrated and evaluated on a real 3-bar tensegrity robot.

[1]  Joran W. Booth,et al.  Sensor Tendons for Soft Robot Shape Estimation , 2022, 2022 IEEE Sensors.

[2]  Kostas E. Bekris,et al.  6N-DoF Pose Tracking for Tensegrity Robots , 2022, ISRR.

[3]  Lawrence Yunliang Chen,et al.  Real2Sim2Real: Self-Supervised Learning of Physical Single-Step Dynamic Actions for Planar Robot Casting , 2021, 2022 International Conference on Robotics and Automation (ICRA).

[4]  J. Tenenbaum,et al.  Contact Points Discovery for Soft-Body Manipulations with Differentiable Physics , 2022, ICLR.

[5]  J. Tenenbaum,et al.  DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools , 2022, ICLR.

[6]  Kostas E. Bekris,et al.  A Recurrent Differentiable Engine for Modeling Tensegrity Robots Trainable with Low-Frequency Data , 2022, 2022 International Conference on Robotics and Automation (ICRA).

[7]  C. K. Liu,et al.  Real-time Model Predictive Control and System Identification Using Differentiable Physics Simulation , 2022, ArXiv.

[8]  Michael C. Yip,et al.  Parameter Identification and Motion Control for Articulated Rigid Body Robots Using Differentiable Position-based Dynamics , 2022, ArXiv.

[9]  Kostas E. Bekris,et al.  CaTGrasp: Learning Category-Level Task-Relevant Grasping in Clutter from Simulation , 2021, 2022 International Conference on Robotics and Automation (ICRA).

[10]  Gaurav S. Sukhatme,et al.  Probabilistic Inference of Simulation Parameters via Parallel Differentiable Simulation , 2021, 2022 International Conference on Robotics and Automation (ICRA).

[11]  Sabrina M. Neuman,et al.  GRiD: GPU-Accelerated Rigid Body Dynamics with Analytical Gradients , 2021, 2022 International Conference on Robotics and Automation (ICRA).

[12]  E. Burdet,et al.  The gap in functional electrical stimulation simulation , 2022 .

[13]  K. Skinner,et al.  Towards Sim2Real for Shipwreck Detection in Side Scan Sonar Imagery , 2022 .

[14]  Sabrina M. Neuman,et al.  Closing the Sim-to-Real Gap for Ultra-Low-Cost, Resource-Constrained, Quadruped Robot Platforms , 2022 .

[15]  Nicholas R. Waytowich,et al.  Toward Real-World Implementation of Deep Reinforcement Learning for Vision-Based Autonomous Drone Navigation with Mission , 2022 .

[16]  V. Koltun,et al.  Differentiable Simulation of Soft Multi-body Systems , 2022, NeurIPS.

[17]  Joran W. Booth,et al.  Integrated Sensing in Robotic Skin Modules , 2021, 2021 IEEE Sensors.

[18]  Kostas E. Bekris,et al.  Tensegrity Robotics. , 2021, Soft robotics.

[19]  Jan Peters,et al.  A Differentiable Newton-Euler Algorithm for Real-World Robotics , 2021, ArXiv.

[20]  Wojciech Matusik,et al.  Underwater Soft Robot Modeling and Control With Differentiable Simulation , 2021, IEEE Robotics and Automation Letters.

[21]  Carmel Majidi,et al.  Numerical Simulation of an Untethered Omni-Directional Star-Shaped Swimming Robot , 2021, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Yashraj S. Narang,et al.  DiSECt: A Differentiable Simulation Engine for Autonomous Robotic Cutting , 2021, Robotics: Science and Systems.

[23]  Moritz Bächer,et al.  Design and Control of Soft Robots Using Differentiable Simulation , 2021, Current Robotics Reports.

[24]  Sanja Fidler,et al.  gradSim: Differentiable simulation for system identification and visuomotor control , 2021, ICLR.

[25]  Kostas E. Bekris,et al.  Sim2Sim Evaluation of a Novel Data-Efficient Differentiable Physics Engine for Tensegrity Robots , 2020, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[26]  Kostas E. Bekris,et al.  Adaptive tensegrity locomotion: Controlling a compliant icosahedron with symmetry-reduced reinforcement learning , 2019, Int. J. Robotics Res..

[27]  C. Karen Liu,et al.  Fast and Feature-Complete Differentiable Physics Engine for Articulated Rigid Bodies with Contact Constraints , 2021, Robotics: Science and Systems.

[28]  Boris Belousov,et al.  Neural Posterior Domain Randomization , 2021, CoRL.

[29]  C. Majidi,et al.  Dynamic simulation of articulated soft robots , 2020, Nature Communications.

[30]  Kostas E. Bekris,et al.  A First Principles Approach for Data-Efficient System Identification of Spring-Rod Systems via Differentiable Physics Engines , 2020, L4DC.

[31]  Wujie Wang,et al.  Differentiable Molecular Simulations for Control and Learning , 2020, ArXiv.

[32]  Vladlen Koltun,et al.  Deep Drone Racing: From Simulation to Reality With Domain Randomization , 2019, IEEE Transactions on Robotics.

[33]  Alberto Rodriguez,et al.  TossingBot: Learning to Throw Arbitrary Objects With Residual Physics , 2019, IEEE Transactions on Robotics.

[34]  Robert E. Skelton,et al.  Design and control of tensegrity morphing airfoils , 2020 .

[35]  Robert E. Skelton,et al.  MOTES: Modeling of Tensegrity Structures , 2019, J. Open Source Softw..

[36]  Carmel Majidi,et al.  On Planar Discrete Elastic Rod Models for the Locomotion of Soft Robots. , 2019, Soft robotics.

[37]  Dieter Fox,et al.  BayesSim: adaptive domain randomization via probabilistic inference for robotics simulators , 2019, Robotics: Science and Systems.

[38]  Gaurav S. Sukhatme,et al.  Interactive Differentiable Simulation , 2019, ArXiv.

[39]  Huseyin Atakan Varol,et al.  Computer Vision-Based Pose Estimation of Tensegrity Robots Using Fiducial Markers , 2019, 2019 IEEE/SICE International Symposium on System Integration (SII).

[40]  Vaishnav Tadiparthi,et al.  STEDY: Software for TEnsegrity DYnamics , 2019, J. Open Source Softw..

[41]  Jürgen Leitner,et al.  Quantifying the Reality Gap in Robotic Manipulation Tasks , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[42]  Alberto Garcia-Garcia,et al.  UnrealROX: an extremely photorealistic virtual reality environment for robotics simulations and synthetic data generation , 2018, Virtual Reality.

[43]  Marc Toussaint,et al.  Differentiable Physics and Stable Modes for Tool-Use and Manipulation Planning , 2018, Robotics: Science and Systems.

[44]  Alice M. Agogino,et al.  Tensegrity Robot Locomotion Under Limited Sensory Inputs via Deep Reinforcement Learning , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[45]  Alice M. Agogino,et al.  Design, Simulation, and Testing of a Flexible Actuated Spine for Quadruped Robots , 2018, 1804.06527.

[46]  Joshua B. Tenenbaum,et al.  End-to-End Differentiable Physics for Learning and Control , 2018, NeurIPS.

[47]  Jonas Degrave,et al.  A DIFFERENTIABLE PHYSICS ENGINE FOR DEEP LEARNING IN ROBOTICS , 2016, Front. Neurorobot..

[48]  Sergey Levine,et al.  Deep reinforcement learning for tensegrity robot locomotion , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[49]  Adrian K. Agogino,et al.  A bio-inspired tensegrity manipulator with multi-DOF, structurally compliant joints , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[50]  Roger D. Quinn,et al.  Towards bridging the reality gap between tensegrity simulation and robotic hardware , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[51]  Benjamin Schrauwen,et al.  Design and control of compliant tensegrity robots through simulation and hardware validation , 2014, Journal of The Royal Society Interface.

[52]  Atil Iscen,et al.  Design and evolution of a modular tensegrity robot platform , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[53]  Alice M. Agogino,et al.  SUPERball : Exploring Tensegrities for Planetary Probes , 2014 .

[54]  Alois Knoll,et al.  CALIPER: A universal robot simulation framework for tendon-driven robots , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[55]  Chandana Paul,et al.  Design and control of tensegrity robots for locomotion , 2006, IEEE Transactions on Robotics.

[56]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.