PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics

Simulated virtual environments serve as one of the main driving forces behind developing and evaluating skill learning algorithms. However, existing environments typically only simulate rigid body physics. Additionally, the simulation process usually does not provide gradients that might be useful for planning and control optimizations. We introduce a new differentiable physics benchmark called PasticineLab, which includes a diverse collection of soft body manipulation tasks. In each task, the agent uses manipulators to deform the plasticine into a desired configuration. The underlying physics engine supports differentiable elastic and plastic deformation using the DiffTaichi system, posing many underexplored challenges to robotic agents. We evaluate several existing reinforcement learning (RL) methods and gradient-based methods on this benchmark. Experimental results suggest that 1) RL-based approaches struggle to solve most of the tasks efficiently; 2) gradient-based approaches, by optimizing open-loop control sequences with the built-in differentiable physics engine, can rapidly find a solution within tens of iterations, but still fall short on multi-stage tasks that require long-term planning. We expect that PlasticineLab will encourage the development of novel algorithms that combine differentiable physics and RL for more complex physics-based skill learning tasks. PlasticineLab is publicly available 1.

[1]  Andrew J. Davison,et al.  Sim-to-Real Reinforcement Learning for Deformable Object Manipulation , 2018, CoRL.

[2]  Joshua B. Tenenbaum,et al.  A Compositional Object-Based Approach to Learning Physical Dynamics , 2016, ICLR.

[3]  Emanuel Todorov,et al.  Trajectory optimization for domains with contacts using inverse dynamics , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Jiancheng Liu,et al.  ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[5]  Herke van Hoof,et al.  Addressing Function Approximation Error in Actor-Critic Methods , 2018, ICML.

[6]  Chenfanfu Jiang,et al.  The material point method for simulating continuum materials , 2016, SIGGRAPH Courses.

[7]  Pieter Abbeel,et al.  Learning to Manipulate Deformable Objects without Demonstrations , 2019, Robotics: Science and Systems.

[8]  Ali Farhadi,et al.  AI2-THOR: An Interactive 3D Environment for Visual AI , 2017, ArXiv.

[9]  Joshua B. Tenenbaum,et al.  The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark Towards Physically Realistic Embodied AI , 2021, 2022 International Conference on Robotics and Automation (ICRA).

[10]  Frédo Durand,et al.  DiffTaichi: Differentiable Programming for Physical Simulation , 2020, ICLR.

[11]  Chenfanfu Jiang,et al.  The affine particle-in-cell method , 2015, ACM Trans. Graph..

[12]  Jonas Degrave,et al.  A DIFFERENTIABLE PHYSICS ENGINE FOR DEEP LEARNING IN ROBOTICS , 2016, Front. Neurorobot..

[13]  Ming C. Lin,et al.  Scalable Differentiable Physics for Learning and Control , 2020, ICML.

[14]  Eftychios Sifakis,et al.  An adaptive generalized interpolation material point method for simulating elastoplastic materials , 2017, ACM Trans. Graph..

[15]  Pieter Abbeel,et al.  Benchmarking Deep Reinforcement Learning for Continuous Control , 2016, ICML.

[16]  Davide Scaramuzza,et al.  Fast Trajectory Optimization for Agile Quadrotor Maneuvers with a Cable-Suspended Payload , 2017, Robotics: Science and Systems.

[17]  Jitendra Malik,et al.  Habitat: A Platform for Embodied AI Research , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[18]  Frédo Durand,et al.  Taichi , 2019, ACM Trans. Graph..

[19]  Eftychios Sifakis,et al.  Computing the Singular Value Decomposition of 3x3 matrices with minimal branching and elementary floating point operations , 2011 .

[20]  Andre Pradhana,et al.  Drucker-prager elastoplasticity for sand animation , 2016, ACM Trans. Graph..

[21]  Marc G. Bellemare,et al.  The Arcade Learning Environment: An Evaluation Platform for General Agents (Extended Abstract) , 2012, IJCAI.

[22]  Ming C. Lin,et al.  Differentiable Cloth Simulation for Inverse Problems , 2019, NeurIPS.

[23]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[24]  Russ Tedrake,et al.  A direct method for trajectory optimization of rigid bodies through contact , 2014, Int. J. Robotics Res..

[25]  Alexey Stomakhin,et al.  A material point method for snow simulation , 2013, ACM Trans. Graph..

[26]  Sanja Fidler,et al.  VirtualHome: Simulating Household Activities Via Programs , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[27]  Roi Poranne,et al.  Trajectory Optimization for Cable-Driven Soft Robot Locomotion , 2019, Robotics: Science and Systems.

[28]  Connor Schenck,et al.  SPNets: Differentiable Fluid Dynamics for Deep Neural Networks , 2018, CoRL.

[29]  Danfei Xu,et al.  Folding deformable objects using predictive simulation and trajectory optimization , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[30]  Gaurav S. Sukhatme,et al.  Interactive Differentiable Simulation , 2019, ArXiv.

[31]  Yang Wang,et al.  Optimizing Intersection-Over-Union in Deep Neural Networks for Image Segmentation , 2016, ISVC.

[32]  Chuang Gan,et al.  CLEVRER: CoLlision Events for Video REpresentation and Reasoning , 2020, ICLR.

[33]  Cosimo Della Santina,et al.  Dynamic control of soft robots interacting with the environment , 2018, 2018 IEEE International Conference on Soft Robotics (RoboSoft).

[34]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[35]  Joshua B. Tenenbaum,et al.  End-to-End Differentiable Physics for Learning and Control , 2018, NeurIPS.

[36]  Vijay Kumar,et al.  Trajectory generation and control of a quadrotor with a cable-suspended load - A differentially-flat hybrid system , 2013, 2013 IEEE International Conference on Robotics and Automation.

[37]  Leonidas J. Guibas,et al.  SAPIEN: A SimulAted Part-Based Interactive ENvironment , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Vijay Kumar,et al.  Mixed Integer Quadratic Program trajectory generation for a quadrotor with a cable-suspended payload , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[39]  Andre Pradhana,et al.  A moving least squares material point method with displacement discontinuity and two-way rigid body coupling , 2018, ACM Trans. Graph..

[40]  Vladlen Koltun,et al.  Learning to Control PDEs with Differentiable Physics , 2020, ICLR.

[41]  Chuang Gan,et al.  ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation , 2020, ArXiv.

[42]  Jiajun Wu,et al.  Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids , 2018, ICLR.

[43]  Jeannette Bohg,et al.  Self-Supervised Learning of State Estimation for Manipulating Deformable Linear Objects , 2020, IEEE Robotics and Automation Letters.

[44]  J. Teran,et al.  Dynamic anticrack propagation in snow , 2018, Nature Communications.

[45]  Daniela Rus,et al.  Dynamics and trajectory optimization for a soft spatial fluidic elastomer manipulator , 2016, Int. J. Robotics Res..

[46]  Sergey Levine,et al.  Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor , 2018, ICML.

[47]  D. Sulsky Erratum: Application of a particle-in-cell method to solid mechanics , 1995 .

[48]  Bo Ren,et al.  Fluid directed rigid body control using deep reinforcement learning , 2018, ACM Trans. Graph..

[49]  Razvan Pascanu,et al.  Interaction Networks for Learning about Objects, Relations and Physics , 2016, NIPS.

[50]  Bernhard Thomaszewski,et al.  ADD , 2020, ACM Trans. Graph..

[51]  Cecilia Laschi,et al.  Control Strategies for Soft Robotic Manipulators: A Survey. , 2018, Soft robotics.

[52]  Daniel L. K. Yamins,et al.  Flexible Neural Representation for Physics Prediction , 2018, NeurIPS.