Automatic Differentiation and Continuous Sensitivity Analysis of Rigid Body Dynamics

A key ingredient to achieving intelligent behavior is physical understanding that equips robots with the ability to reason about the effects of their actions in a dynamic environment. Several methods have been proposed to learn dynamics models from data that inform model-based control algorithms. While such learning-based approaches can model locally observed behaviors, they fail to generalize to more complex dynamics and under long time horizons. In this work, we introduce a differentiable physics simulator for rigid body dynamics. Leveraging various techniques for differential equation integration and gradient calculation, we compare different methods for parameter estimation that allow us to infer the simulation parameters that are relevant to estimation and control of physical systems. In the context of trajectory optimization, we introduce a closed-loop model-predictive control algorithm that infers the simulation parameters through experience while achieving cost-minimizing performance.

[1]  Nicolas Mansard,et al.  Analytical Derivatives of Rigid Body Dynamics Algorithms , 2018, Robotics: Science and Systems.

[2]  Dieter Fox,et al.  Gaussian Processes and Reinforcement Learning for Identification and Control of an Autonomous Blimp , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[3]  Bob Carpenter,et al.  The Stan Math Library: Reverse-Mode Automatic Differentiation in C++ , 2015, ArXiv.

[4]  Radu Serban,et al.  User Documentation for CVODES: An ODE Solver with Sensitivity Analysis Capabilities , 2002 .

[5]  Martin A. Riedmiller,et al.  Approximate real-time optimal control based on sparse Gaussian process models , 2014, 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[6]  C. Rasmussen,et al.  Improving PILCO with Bayesian Neural Network Dynamics Models , 2016 .

[7]  Patrick MacAlpine,et al.  Humanoid robots learning to walk faster: from the real world to simulation and back , 2013, AAMAS.

[8]  Wei Chen,et al.  Learning to predict the cosmological structure formation , 2018, Proceedings of the National Academy of Sciences.

[9]  Jiajun Wu,et al.  Galileo: Perceiving Physical Object Properties by Integrating a Physics Engine with Deep Learning , 2015, NIPS.

[10]  J. Zico Kolter,et al.  OptNet: Differentiable Optimization as a Layer in Neural Networks , 2017, ICML.

[11]  M. L. Chambers.  The Mathematical Theory of Optimal Processes , 1965 .

[12]  Vaibhav Dixit,et al.  A Comparison of Automatic Differentiation and Continuous Sensitivity Analysis for Derivatives of Differential Equation Solutions , 2018, 2021 IEEE High Performance Extreme Computing Conference (HPEC).

[13]  James M. Rehg,et al.  Aggressive driving with model predictive path integral control , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Kostas E. Bekris,et al.  Fast Model Identification via Physics Engines for Data-Efficient Policy Search , 2017, IJCAI.

[15]  Andrew W. Moore,et al.  Fast, Robust Adaptive Control by Learning only Forward Models , 1991, NIPS.

[16]  Darwin G. Caldwell,et al.  RobCoGen: a code generator for efficient kinematics and dynamics of articulated robots, based on Domain Specific Languages , 2016 .

[17]  Sergey Levine,et al.  Unsupervised Learning for Physical Interaction through Video Prediction , 2016, NIPS.

[18]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Finale Doshi-Velez,et al.  Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networks , 2016, ICLR.

[20]  Sergey Levine,et al.  One-shot learning of manipulation skills with online dynamics adaptation and neural network priors , 2015, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  Twan Koolen,et al.  Julia for robotics: simulation and real-time control in a high-level programming language , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[22]  Yevgen Chebotar,et al.  Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[23]  Yuval Tassa,et al.  DeepMind Control Suite , 2018, ArXiv.

[24]  Jiajun Wu,et al.  Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids , 2018, ICLR.

[25]  J. Denavit,et al.  A kinematic notation for lower pair mechanisms based on matrices , 1955 .

[26]  Sergey Levine,et al.  Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models , 2018, NeurIPS.

[27]  Jason Yosinski,et al.  Hamiltonian Neural Networks , 2019, NeurIPS.

[28]  Joshua B. Tenenbaum,et al.  A Compositional Object-Based Approach to Learning Physical Dynamics , 2016, ICLR.

[29]  Jonas Degrave,et al.  A DIFFERENTIABLE PHYSICS ENGINE FOR DEEP LEARNING IN ROBOTICS , 2016, Front. Neurorobot..

[30]  Connor Schenck,et al.  SPNets: Differentiable Fluid Dynamics for Deep Neural Networks , 2018, CoRL.

[31]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[32]  Daniel L. K. Yamins,et al.  Flexible Neural Representation for Physics Prediction , 2018, NeurIPS.

[33]  L. S. Pontryagin,et al.  Mathematical Theory of Optimal Processes , 1962 .

[34]  Andrew W. Moore,et al.  Locally Weighted Learning for Control , 1997, Artificial Intelligence Review.

[35]  Zhijian Liu,et al.  Modeling Parts, Structure, and System Dynamics via Predictive Learning , 2019 .

[36]  Carl E. Rasmussen,et al.  PILCO: A Model-Based and Data-Efficient Approach to Policy Search , 2011, ICML.

[37]  Christopher G. Atkeson,et al.  Neural networks and differential dynamic programming for reinforcement learning problems , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[38]  Jiajun Wu,et al.  Physics 101: Learning Physical Object Properties from Unlabeled Videos , 2016, BMVC.

[39]  M. Jerrell.  Automatic Differentiation and Interval Arithmetic for Estimation of Disequilibrium Models , 1997 .

[40]  Raia Hadsell,et al.  Graph networks as learnable physics engines for inference and control , 2018, ICML.

[41]  Emanuel Todorov,et al.  Physically consistent state estimation and system identification for contacts , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[42]  S. Levine,et al.  Reasoning About Physical Interactions with Object-Centric Models , 2018 .

[43]  Hessam Babaee,et al.  Deep Learning of Turbulent Scalar Mixing , 2018, Physical Review Fluids.

[44]  Jiajun Wu,et al.  DensePhysNet: Learning Dense Physical Object Representations via Multi-step Dynamic Interactions , 2019, Robotics: Science and Systems.

[45]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[46]  Joshua B. Tenenbaum,et al.  End-to-End Differentiable Physics for Learning and Control , 2018, NeurIPS.

[47]  Athanasios S. Polydoros,et al.  Survey of Model-Based Reinforcement Learning: Applications on Robotics , 2017, J. Intell. Robotic Syst..

[48]  Jonas Buchli,et al.  Automatic Differentiation of Rigid Body Dynamics for Optimal Control and Estimation , 2017, Adv. Robotics.

[49]  Emanuel Todorov,et al.  Iterative Linear Quadratic Regulator Design for Nonlinear Biological Movement Systems , 2004, ICINCO.

[50]  Jiajun Wu,et al.  Learning to See Physics via Visual De-animation , 2017, NIPS.

[51]  Roy Featherstone,et al.  Rigid Body Dynamics Algorithms , 2007 .

[52]  Franciso-Javier Montecillo-Puente.  A Dynamic Simulator for Humanoid Robots , 2011 .

[53]  Jan Peters,et al.  Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning , 2019, ICLR.

[54]  Razvan Pascanu,et al.  Interaction Networks for Learning about Objects, Relations and Physics , 2016, NIPS.