Perspectives on Sim2Real Transfer for Robotics: A Summary of the R: SS 2020 Workshop

This report presents the debates, posters, and discussions of the Sim2Real workshop held in conjunction with the 2020 edition of the "Robotics: Science and System" conference. Twelve leaders of the field took competing debate positions on the definition, viability, and importance of transferring skills from simulation to the real world in the context of robotics problems. The debaters also joined a large panel discussion, answering audience questions and outlining the future of Sim2Real in robotics. Furthermore, we invited extended abstracts to this workshop which are summarized in this report. Based on the workshop, this report concludes with directions for practitioners exploiting this technology and for researchers further exploring open problems in this area.

[1]  Sharath Chandra Raparthy,et al.  CuNAS-CUriosity-driven Neural-Augmented Simulator , 2010 .

[2]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[3]  Marcin Andrychowicz,et al.  Solving Rubik's Cube with a Robot Hand , 2019, ArXiv.

[4]  Ken Goldberg,et al.  Learning ambidextrous robot grasping policies , 2019, Science Robotics.

[5]  Joonho Lee,et al.  Learning agile and dynamic motor skills for legged robots , 2019, Science Robotics.

[6]  Yashraj S. Narang,et al.  Inferring the Material Properties of Granular Media for Robotic Tasks , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Gaurav S. Sukhatme,et al.  Augmenting Differentiable Simulators with Neural Networks to Close the Sim2Real Gap , 2020, ArXiv.

[8]  S. Levine,et al.  Learning Agile Robotic Locomotion Skills by Imitating Animals , 2020, Robotics: Science and Systems.

[9]  Vladlen Koltun,et al.  Deep Drone Acrobatics , 2020, Robotics: Science and Systems.

[10]  Akshara Rai,et al.  How to Sim2Real with Gaussian Processes: Prior Mean versus Kernels as Priors , 2020 .

[11]  Rika Antonova,et al.  Modular Latent Space Transfer withAnalytic Manifold Learning , 2020 .

[12]  Kefang Zhang,et al.  Sim2Real Learning of Vision-Based Obstacle Avoidance for Robotic Manipulators , 2020 .

[13]  Mabel M. Zhang.  Necessity for More Realistic Contact Simulation , 2020 .

[14]  Jorge Pena Queralta,et al.  Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey , 2020, 2020 IEEE Symposium Series on Computational Intelligence (SSCI).

[15]  Pieter Abbeel,et al.  Learning to Manipulate Deformable Objects without Demonstrations , 2019, Robotics: Science and Systems.

[16]  Josip Josifovski,et al.  Robust Sim2Real Transfer by Learning Inverse Dynamics of Simulated Systems , 2020 .

[17]  M. Sundermeyer,et al.  BlenderProc: Reducing the Reality Gap with Photorealistic Rendering , 2020 .

[18]  Sonia Chernova,et al.  Sim2Real Predictivity: Does Evaluation in Simulation Predict Real-World Performance? , 2019, IEEE Robotics and Automation Letters.

[19]  L. Paull,et al.  On Assessing the Value of Simulation for Robotics , 2020 .

[20]  Josiah P. Hanna,et al.  An Imitation from Observation Approach to Sim-to-Real Transfer , 2020, ArXiv.

[21]  Fei Chen,et al.  The Importance and the Limitations of Sim2Real for Robotic Manipulation in Precision Agriculture , 2020, ArXiv.

[22]  D. Fox,et al.  Online BayesSim for Combined Simulator Parameter Inference and Policy Improvement , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  Kevin R. Green,et al.  Learning to Walk without Dynamics Randomization , 2020 .

[24]  Pieter Abbeel,et al.  Learning Predictive Representations for Deformable Objects Using Contrastive Estimation , 2020, CoRL.

[25]  Josip Josifovski,et al.  Continual Learning on Incremental Simulations for Real-World Robotic Manipulation Tasks , 2020 .

[26]  Mohi Khansari,et al.  RL-CycleGAN: Reinforcement Learning Aware Simulation-to-Real , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  D. Manocha,et al.  Accurate High Fidelity Simulations for Training Robot Navigation Policies for Dense Crowds using Deep Reinforcement Learning , 2020 .