Structured learning of rigid‐body dynamics: A survey and unified view from a robotics perspective

Accurate models of mechanical system dynamics are often critical for model-based control and reinforcement learning. Fully data-driven dynamics models promise to ease the process of modeling and analysis, but require considerable amounts of data for training and often do not generalize well to unseen parts of the state space. Combining data-driven modelling with prior analytical knowledge is an attractive alternative as the inclusion of structural knowledge into a regression model improves the model's data efficiency and physical integrity. In this article, we survey supervised regression models that combine rigid-body mechanics with data-driven modelling techniques. We analyze the different latent functions (such as kinetic energy or dissipative forces) and operators (such as differential operators and projection matrices) underlying common descriptions of rigid-body mechanics. Based on this analysis, we provide a unified view on the combination of data-driven regression models, such as neural networks and Gaussian processes, with analytical model priors. Further, we review and discuss key techniques for designing structured models such as automatic differentiation.

[1]  Carl E. Rasmussen,et al.  PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaos , 2019, ICML.

[2]  Stefen Hui,et al.  Application of feedforward neural networks to dynamical system identification and control , 1993, IEEE Trans. Control. Syst. Technol..

[3]  G. Oriolo,et al.  Robotics: Modelling, Planning and Control , 2008 .

[4]  David Duvenaud,et al.  Scalable Gradients for Stochastic Differential Equations , 2020, AISTATS.

[5]  Michael I. Jordan.  Constrained supervised learning , 1992 .

[6]  Mykel J. Kochenderfer,et al.  Deep Dynamical Modeling and Control of Unsteady Fluid Flows , 2018, NeurIPS.

[7]  Andrew Jaegle,et al.  Hamiltonian Generative Networks , 2020, ICLR.

[8]  Neil D. Lawrence,et al.  Latent Autoregressive Gaussian Processes Models for Robust System Identification , 2016 .

[9]  Lorenz Wellhausen,et al.  Learning quadrupedal locomotion over challenging terrain , 2020, Science Robotics.

[10]  R. Kalaba,et al.  Analytical Dynamics: A New Approach , 1996 .

[11]  Sami Haddadin,et al.  First-order-principles-based constructive network topologies: An application to robot inverse dynamics , 2017, 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids).

[12]  Philip Rabinowitz,et al.  Methods of Numerical Integration , 1985 .

[13]  David Duvenaud,et al.  Probabilistic ODE Solvers with Runge-Kutta Means , 2014, NIPS.

[14]  Sebastian Trimpe,et al.  Actively Learning Gaussian Process Dynamics , 2019, L4DC.

[15]  Sandra Hirche,et al.  Learning stable Gaussian process state space models , 2017, 2017 American Control Conference (ACC).

[16]  Carl E. Rasmussen,et al.  Bayesian Inference and Learning in Gaussian Process State-Space Models with Particle MCMC , 2013, NIPS.

[17]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[18]  Markus Heinonen,et al.  LEARNING STOCHASTIC DIFFERENTIAL EQUATIONS WITH GAUSSIAN PROCESSES WITHOUT GRADIENT MATCHING , 2018, 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP).

[19]  Yuichi Nakamura,et al.  Approximation of dynamical systems by continuous time recurrent neural networks , 1993, Neural Networks.

[20]  Felix Berkenkamp,et al.  Structured Variational Inference in Partially Observable UnstableGaussian Process State Space Models , 2020, L4DC.

[21]  Adrien Escande,et al.  Identification of fully physical consistent inertial parameters using optimization on manifolds , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[22]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[23]  Thomas B. Schön,et al.  System identification of nonlinear state-space models , 2011, Autom..

[24]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[25]  Neil D. Lawrence,et al.  Kernels for Vector-Valued Functions: a Review , 2011, Found. Trends Mach. Learn..

[26]  Thomas B. Schön,et al.  Linearly constrained Gaussian processes , 2017, NIPS.

[27]  Thomas R. Kane,et al.  THEORY AND APPLICATIONS , 1984 .

[28]  Kim D. Listmann,et al.  Deep Lagrangian Networks for end-to-end learning of energy-based control for under-actuated systems , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[29]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[30]  Firdaus E. Udwadia,et al.  Nonideal Constraints and Lagrangian Dynamics , 2000 .

[31]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[32]  Jung-Min Park,et al.  Independent Joint Learning: A novel task-to-task transfer learning scheme for robot models , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[33]  Firdaus E. Udwadia,et al.  Equations of motion for constrained mechanical systems and the extended D'Alembert's principle , 1997 .

[34]  Jakub W. Pachocki,et al.  Dota 2 with Large Scale Deep Reinforcement Learning , 2019, ArXiv.

[35]  Jan Peters,et al.  Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning , 2019, ICLR.

[36]  Agathe Girard,et al.  Dynamic systems identification with Gaussian processes , 2005 .

[37]  G. Martin,et al.  Nonlinear model predictive control , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[38]  Lennart Ljung,et al.  Nonlinear System Identification: A User-Oriented Road Map , 2019, IEEE Control Systems.

[39]  Jan Peters,et al.  A Differentiable Newton Euler Algorithm for Multi-body Model Learning , 2020, ArXiv.

[40]  Michael I. Jordan,et al.  Learning Without Mixing: Towards A Sharp Analysis of Linear System Identification , 2018, COLT.

[41]  Markus Heinonen,et al.  Learning unknown ODE models with Gaussian processes , 2018, ICML.

[42]  Bernhard Schölkopf,et al.  Statistical Learning Theory: Models, Concepts, and Results , 2008, Inductive Logic.

[43]  Phailaung Phohomsiri,et al.  Explicit equations of motion for constrained mechanical systems with singular mass matrices and applications to multi-body dynamics , 2006, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[44]  Sami Haddadin,et al.  FOP Networks for Learning Humanoid Body Schema and Dynamics , 2018, 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids).

[45]  Nan Rong,et al.  What makes some POMDP problems easy to approximate? , 2007, NIPS.

[46]  A. R. Geist,et al.  Learning Constrained Dynamics with Gauss Principle adhering Gaussian Processes , 2020, L4DC.

[47]  Han-Pang Huang,et al.  Learning the Inverse Dynamics of Robotic Manipulators in Structured Reproducing Kernel Hilbert Space , 2016, IEEE Transactions on Cybernetics.

[48]  Barak A. Pearlmutter,et al.  Automatic differentiation in machine learning: a survey , 2015, J. Mach. Learn. Res..

[49]  Marco Hutter,et al.  Per-Contact Iteration Method for Solving Contact Dynamics , 2018, IEEE Robotics and Automation Letters.

[50]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[51]  Miles Cranmer,et al.  Lagrangian Neural Networks , 2020, ICLR 2020.

[52]  C. F. Gauss,et al.  Über Ein Neues Allgemeines Grundgesetz der Mechanik , 1829 .

[53]  Dana Kulic,et al.  Online Incremental Learning of Inverse Dynamics Incorporating Prior Knowledge , 2011, AIS.

[54]  Farhad Aghili,et al.  A unified approach for inverse and direct dynamics of constrained multibody systems based on linear projection operator: applications to control and simulation , 2005, IEEE Transactions on Robotics.

[55]  Jun Nakanishi,et al.  A Bayesian Approach to Nonlinear Parameter Identification for Rigid Body Dynamics , 2006, Robotics: Science and Systems.

[56]  Ludovic Righetti,et al.  An Open Torque-Controlled Modular Robot Architecture for Legged Locomotion Research , 2019, IEEE Robotics and Automation Letters.

[57]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[58]  Jason Yosinski,et al.  Hamiltonian Neural Networks , 2019, NeurIPS.

[59]  David Duvenaud,et al.  Automatic model construction with Gaussian processes , 2014 .

[60]  Simo Särkkä,et al.  A probabilistic model for the numerical solution of initial value problems , 2016, Statistics and Computing.

[61]  Philippe Wenk,et al.  ODIN: ODE-Informed Regression for Parameter and State Inference in Time-Continuous Dynamical Systems , 2019, AAAI.

[62]  J. Y. S. Luh,et al.  On-Line Computational Scheme for Mechanical Manipulators , 1980 .

[63]  Roy Featherstone,et al.  Rigid Body Dynamics Algorithms , 2007 .

[64]  Amir Ali Ahmadi,et al.  Learning Dynamical Systems with Side Information , 2020, L4DC.

[65]  Jean-Jacques E. Slotine,et al.  Linear Matrix Inequalities for Physically Consistent Inertial Parameter Identification: A Statistical Perspective on the Mass Distribution , 2017, IEEE Robotics and Automation Letters.

[66]  Ruben Grandia,et al.  Contact Invariant Model Learning for Legged Robot Locomotion , 2018, IEEE Robotics and Automation Letters.

[67]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[68]  Thomas B. Schön,et al.  Linearly Constrained Neural Networks , 2020, ArXiv.

[69]  Christoph H. Lampert,et al.  Learning Equations for Extrapolation and Control , 2018, ICML.

[70]  Randal W. Beard.  Linear operator equations with applications in control and signal processing , 2002 .

[71]  R. Kalaba,et al.  A new perspective on constrained motion , 1992, Proceedings of the Royal Society of London. Series A: Mathematical and Physical Sciences.

[72]  Sergey Levine,et al.  Deep Dynamics Models for Learning Dexterous Manipulation , 2019, CoRL.

[73]  Junggon Kim.  Lie Group Formulation of Articulated Rigid Body Dynamics , 2012 .

[74]  Jun Nakanishi,et al.  A unifying framework for robot control with redundant DOFs , 2007, Auton. Robots.

[75]  James Hensman,et al.  Identification of Gaussian Process State Space Models , 2017, NIPS.

[76]  M. McCall,et al.  Rigid Body Dynamics , 2008 .

[77]  Christopher G. Atkeson,et al.  Estimation of Inertial Parameters of Manipulator Loads and Links , 1986 .

[78]  Frank Kirchner,et al.  An Analytical and Modular Software Workbench for Solving Kinematics and Dynamics of Series-Parallel Hybrid Robots , 2019, Journal of Mechanisms and Robotics.

[79]  Mykel J. Kochenderfer,et al.  Structured Mechanical Models for Robot Learning and Control , 2020, L4DC.

[80]  Dana Kulic,et al.  Stable Gaussian process based tracking control of Lagrangian systems , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[81]  Shivesh Kumar,et al.  Modular and Analytical Methods for Solving Kinematics and Dynamics of Series-Parallel Hybrid Robots (Modulare und analytische Verfahren zur Lösung von Kinematik und Dynamik von seriell-parallelen Hybridrobotern) , 2019 .

[82]  Sergey Levine,et al.  Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models , 2018, NeurIPS.

[83]  Austin Wang,et al.  Encoding Physical Constraints in Differentiable Newton-Euler Algorithm , 2020, L4DC.

[84]  K. Jarrod Millman,et al.  Array programming with NumPy , 2020, Nat..

[85]  Olivier Stasse,et al.  The Pinocchio C++ library : A fast and flexible implementation of rigid body dynamics algorithms and their analytical derivatives , 2019, 2019 IEEE/SICE International Symposium on System Integration (SII).

[86]  C. Atkeson,et al.  Estimation of inertial parameters of rigid body links of manipulators , 1985, 1985 24th IEEE Conference on Decision and Control.

[87]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[88]  Karl H. Johansson,et al.  Identifying Causal Structure in Dynamical Systems , 2020, ArXiv.

[89]  Jorge Nocedal,et al.  Optimization Methods for Large-Scale Machine Learning , 2016, SIAM Rev..

[90]  Joonho Lee,et al.  Learning agile and dynamic motor skills for legged robots , 2019, Science Robotics.

[91]  Giorgio Metta,et al.  Incremental semiparametric inverse dynamics learning , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[92]  Duy Nguyen-Tuong,et al.  Optimizing Long-term Predictions for Model-based Policy Search , 2017, CoRL.

[93]  O. Nelles.  Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuzzy Models , 2000 .

[94]  Jan Peters,et al.  Model learning for robot control: a survey , 2011, Cognitive Processing.

[95]  A. Borisov,et al.  Rigid Body Dynamics , 2018 .

[96]  R. Kalaba,et al.  On the foundations of analytical dynamics , 2002 .

[97]  Markus Lange-Hegermann,et al.  Algorithmic Linearly Constrained Gaussian Processes , 2018, NeurIPS.

[98]  Jan Peters,et al.  Using model knowledge for learning inverse dynamics , 2010, 2010 IEEE International Conference on Robotics and Automation.

[99]  Carl E. Rasmussen,et al.  PILCO: A Model-Based and Data-Efficient Approach to Policy Search , 2011, ICML.

[100]  Duy Nguyen-Tuong,et al.  Probabilistic Recurrent State-Space Models , 2018, ICML.

[101]  George F. Corliss,et al.  Applications of differentiation arithmetic , 1988 .

[102]  Firdaus E. Udwadia,et al.  Unified Approach to Modeling and Control of Rigid Multibody Systems , 2016 .

[103]  M. Jansen.  Learning an Accurate Neural Model of the Dynamics of a Typical Industrial Robot , 1994 .