Scalable sim-to-real transfer of soft robot designs

The manual design of soft robots and their controllers is notoriously challenging, but it could be augmented—or, in some cases, entirely replaced—by automated design tools. Machine learning algorithms can automatically propose, test, and refine designs in simulation, and the most promising ones can then be manufactured in reality (sim2real). However, it is currently not known how to guarantee that behavior generated in simulation can be preserved when deployed in reality. Although many previous studies have devised training protocols that facilitate sim2real transfer of control polices, little to no work has investigated the simulation-reality gap as a function of morphology. This is due in part to an overall lack of tools capable of systematically designing and rapidly manufacturing robots. Here we introduce a low cost, open source, and modular soft robot design and construction kit, and use it to simulate, fabricate, and measure the simulation-reality gap of minimally complex yet soft, locomoting machines. We prove the scalability of this approach by transferring an order of magnitude more robot designs from simulation to reality than any other method. The kit and its instructions can be found here: github.com/skriegman/sim2real4designs

[1]  Josh C. Bongard,et al.  Automated shapeshifting for function recovery in damaged robots , 2019, Robotics: Science and Systems.

[2]  Razvan Pascanu,et al.  Sim-to-Real Robot Learning from Pixels with Progressive Nets , 2016, CoRL.

[3]  Stéphane Doncieux,et al.  The Transferability Approach: Crossing the Reality Gap in Evolutionary Robotics , 2013, IEEE Transactions on Evolutionary Computation.

[4]  Joonho Lee,et al.  Learning agile and dynamic motor skills for legged robots , 2019, Science Robotics.

[5]  Michael Levin,et al.  Reprogramming cells and tissue patterning via bioelectrical pathways: molecular mechanisms and biomedical opportunities , 2013, Wiley interdisciplinary reviews. Systems biology and medicine.

[6]  Vikash Kumar,et al.  Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Real , 2019, CoRL.

[7]  Wojciech Zaremba,et al.  Domain randomization for transferring deep neural networks from simulation to the real world , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[8]  Charles E. Taylor,et al.  Selection for Wandering Behavior in a Small Robot , 1994, Artificial Life.

[9]  Giovanni Pezzulo,et al.  Top-down models in biology: explanation and control of complex living systems above the molecular level , 2016, Journal of The Royal Society Interface.

[10]  LipsonHod,et al.  Dynamic Simulation of Soft Multimaterial 3D-Printed Objects , 2014 .

[11]  M. Levin.  The Computational Boundary of a “Self”: Developmental Bioelectricity Drives Multicellularity and Scale-Free Cognition , 2019, Front. Psychol..

[12]  Atil Iscen,et al.  Sim-to-Real: Learning Agile Locomotion For Quadruped Robots , 2018, Robotics: Science and Systems.

[13]  Marcin Andrychowicz,et al.  Sim-to-Real Transfer of Robotic Control with Dynamics Randomization , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Yura N. Perov,et al.  Gait optimization for roombots modular robots — Matching simulation and reality , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Derek Nowrouzezahrai,et al.  Subspace neural physics: fast data-driven interactive simulation , 2019, Symposium on Computer Animation.

[16]  Benjamin Schrauwen,et al.  Design and control of compliant tensegrity robots through simulation and hardware validation , 2014, Journal of The Royal Society Interface.

[17]  Marcin Andrychowicz,et al.  Asymmetric Actor Critic for Image-Based Robot Learning , 2017, Robotics: Science and Systems.

[18]  Inman Harvey,et al.  Noise and the Reality Gap: The Use of Simulation in Evolutionary Robotics , 1995, ECAL.

[19]  Hod Lipson,et al.  Automatic Design and Manufacture of Soft Robots , 2012, IEEE Transactions on Robotics.

[20]  Michael Levin,et al.  Morphogenetic fields in embryogenesis, regeneration, and cancer: Non-local control of complex patterning , 2012, Biosyst..

[21]  Robert J. Wood,et al.  Influence of surface traction on soft robot undulation , 2013, Int. J. Robotics Res..

[22]  Sam Kriegman.  Why virtual creatures matter , 2019 .

[23]  Hod Lipson,et al.  Molecubes: An Open-Source Modular Robotics Kit , 2007 .

[24]  Stefan Jeschke,et al.  Non-smooth Newton Methods for Deformable Multi-body Dynamics , 2019, ACM Trans. Graph..

[25]  Josh Bongard,et al.  A scalable pipeline for designing reconfigurable organisms , 2020, Proceedings of the National Academy of Sciences.

[26]  Inman Harvey,et al.  Evolutionary robotics: the Sussex approach , 1997, Robotics Auton. Syst..

[27]  Antoine Cully,et al.  Robots that can adapt like animals , 2014, Nature.

[28]  Marcin Andrychowicz,et al.  Solving Rubik's Cube with a Robot Hand , 2019, ArXiv.

[29]  Pierre-Yves Oudeyer,et al.  Sim-to-Real Transfer with Neural-Augmented Robot Simulation , 2018, CoRL.

[30]  Sebastian Risi,et al.  1D Printing of Recyclable Robots , 2017, IEEE Robotics and Automation Letters.

[31]  Jordan B. Pollack,et al.  Automatic design and manufacture of robotic lifeforms , 2000, Nature.

[32]  G. Pezzulo,et al.  Re-membering the body: applications of computational neuroscience to the top-down control of regeneration of limbs and other complex organs. , 2015, Integrative biology : quantitative biosciences from nano to macro.

[33]  Hod Lipson,et al.  Resilient Machines Through Continuous Self-Modeling , 2006, Science.

[34]  Andrew J. Davison,et al.  Sim-to-Real Reinforcement Learning for Deformable Object Manipulation , 2018, CoRL.

[35]  Michael Levin,et al.  A linear-encoding model explains the variability of the target morphology in regeneration , 2014, Journal of The Royal Society Interface.