Learning to Control PDEs with Differentiable Physics

Predicting outcomes and planning interactions with the physical world are long-standing goals for machine learning. A variety of such tasks involves continuous physical systems, which can be described by partial differential equations (PDEs) with many degrees of freedom. Existing methods that aim to control the dynamics of such systems are typically limited to relatively short time frames or a small number of interaction parameters. We show that by using a differentiable PDE solver in conjunction with a novel predictor-corrector scheme, we can train neural networks to understand and control complex nonlinear physical systems over long time frames. We demonstrate that our method successfully develops an understanding of complex physical systems and learns to control them for tasks involving multiple PDEs, including the incompressible Navier-Stokes equations.

[1]  Vladlen Koltun,et al.  Lagrangian Fluid Simulation with Continuous Convolutions , 2020, ICLR.

[2]  G. Sell,et al.  Inertial manifolds for nonlinear evolutionary equations , 1988 .

[3]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[4]  C. Hargraves,et al.  DIRECT TRAJECTORY OPTIMIZATION USING NONLINEAR PROGRAMMING AND COLLOCATION , 1987 .

[5]  O. Pironneau.  On optimum design in fluid mechanics , 1974 .

[6]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[7]  Jonas Degrave,et al.  A DIFFERENTIABLE PHYSICS ENGINE FOR DEEP LEARNING IN ROBOTICS , 2016, Front. Neurorobot..

[8]  Steven M. Seitz,et al.  Interactive manipulation of rigid body simulations , 2000, SIGGRAPH.

[9]  Daniel L. K. Yamins,et al.  Flexible Neural Representation for Physics Prediction , 2018, NeurIPS.

[10]  George E. Karniadakis,et al.  Hidden Fluid Mechanics: A Navier-Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data , 2018, ArXiv.

[11]  Joshua B. Tenenbaum,et al.  End-to-End Differentiable Physics for Learning and Control , 2018, NeurIPS.

[12]  J. Templeton,et al.  Reynolds averaged turbulence modelling using deep neural networks with embedded invariance , 2016, Journal of Fluid Mechanics.

[13]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[14]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[15]  G. Batchelor,et al.  An Introduction to Fluid Dynamics , 1968 .

[16]  Stefano Ermon,et al.  Learning Neural PDE Solvers with Convergence Guarantees , 2019, ICLR.

[17]  Ming C. Lin,et al.  Differentiable Cloth Simulation for Inverse Problems , 2019, NeurIPS.

[18]  Niles A. Pierce,et al.  An Introduction to the Adjoint Approach to Design , 2000 .

[19]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  David A. Ham,et al.  Automated Derivation of the Adjoint of High-Level Transient Finite Element Programs , 2013, SIAM J. Sci. Comput..

[21]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[22]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[23]  Andreas Griewank,et al.  Algorithm 755: ADOL-C: a package for the automatic differentiation of algorithms written in C/C++ , 1996, TOMS.

[24]  M. L. Chambers.  The Mathematical Theory of Optimal Processes , 1965 .

[25]  Debora S. Marks,et al.  Learning Protein Structure with a Differentiable Simulator , 2018, ICLR.

[26]  Daniel L. K. Yamins,et al.  Learning to Play with Intrinsically-Motivated Self-Aware Agents , 2018, NeurIPS.

[27]  Ken Perlin,et al.  Accelerating Eulerian Fluid Simulation With Convolutional Networks , 2016, ICML.

[28]  Marc Toussaint,et al.  Differentiable Physics and Stable Modes for Tool-Use and Manipulation Planning , 2018, Robotics: Science and Systems.

[29]  Z. Popovic,et al.  Fluid control using the adjoint method , 2004, SIGGRAPH 2004.

[30]  Jitendra Malik,et al.  Learning to Poke by Poking: Experiential Learning of Intuitive Physics , 2016, NIPS.

[31]  Jiajun Wu,et al.  Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids , 2018, ICLR.

[32]  Dieter Kraft,et al.  On Converting Optimal Control Problems into Nonlinear Programming Problems , 1985 .

[33]  Takuya Akiba,et al.  Chainer: A Deep Learning Framework for Accelerating the Research Cycle , 2019, KDD.

[34]  A. Chorin.  The Numerical Solution of the Navier-Stokes Equations for an Incompressible Fluid , 2015 .

[35]  Barbara Solenthaler,et al.  Data-driven fluid simulations using regression forests , 2015, ACM Trans. Graph..

[36]  Gene H. Golub,et al.  Matrix computations , 1983 .

[37]  J. Doyle,et al.  Robust and optimal control , 1995, Proceedings of 35th IEEE Conference on Decision and Control.

[38]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[39]  Bin Dong,et al.  PDE-Net: Learning PDEs from Data , 2017, ICML.

[40]  Nils Thürey,et al.  tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow , 2018, ACM Trans. Graph..

[41]  Arthur Mensch,et al.  Differentiable Dynamic Programming for Structured Prediction and Attention , 2018, ICML.

[42]  Matt J. Kusner,et al.  A Generative Model For Electron Paths , 2018, ICLR.

[43]  William H. Press,et al.  Numerical recipes , 1990 .

[44]  J. Zico Kolter,et al.  OptNet: Differentiable Optimization as a Layer in Neural Networks , 2017, ICML.

[45]  Dan Moldovan,et al.  Tangent: Automatic differentiation using source-code transformation for dynamically typed array programming , 2018, NeurIPS.

[46]  H. Bock,et al.  A Multiple Shooting Algorithm for Direct Solution of Optimal Control Problems , 1984 .

[47]  Antony Jameson,et al.  Aerodynamic design via control theory , 1988, J. Sci. Comput..

[48]  Edriss S. Titi,et al.  On approximate Inertial Manifolds to the Navier-Stokes equations , 1990 .

[49]  Sergey Levine,et al.  Unsupervised Learning for Physical Interaction through Video Prediction , 2016, NIPS.

[50]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[51]  Stephan Hoyer,et al.  Learning data-driven discretizations for partial differential equations , 2018, Proceedings of the National Academy of Sciences.

[52]  Thomas Bewley,et al.  Flow control: new challenges for a new Renaissance , 2001 .

[53]  Mykel J. Kochenderfer,et al.  Deep Dynamical Modeling and Control of Unsteady Fluid Flows , 2018, NeurIPS.

[54]  Jiancheng Liu,et al.  ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[55]  Johannes P. Schlöder,et al.  An efficient multiple shooting based reduced SQP strategy for large-scale dynamic process optimization. Part 1: theoretical aspects , 2003, Comput. Chem. Eng..

[56]  Jessica B. Hamrick,et al.  Simulation as an engine of physical scene understanding , 2013, Proceedings of the National Academy of Sciences.

[57]  F. Harlow,et al.  Numerical Calculation of Time‐Dependent Viscous Incompressible Flow of Fluid with Free Surface , 1965 .

[58]  Connor Schenck,et al.  SPNets: Differentiable Fluid Dynamics for Deep Neural Networks , 2018, CoRL.

[59]  Jos Stam,et al.  Stable fluids , 1999, SIGGRAPH.

[60]  Wei Zhang,et al.  Addendum to "Optimal control of multiscale systems using reduced-order models" , 2017 .

[61]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[62]  R. LeVeque.  Numerical methods for conservation laws , 1990 .

[63]  J. Lions.  Optimal Control of Systems Governed by Partial Differential Equations , 1971 .

[64]  Richard Courant,et al.  Methods of Mathematical Physics, Volume II: Partial Differential Equations , 1963 .

[65]  John Sibert,et al.  AD Model Builder: using automatic differentiation for statistical inference of highly parameterized complex nonlinear models , 2012, Optim. Methods Softw..

[66]  Andrew Lewis,et al.  Model reduction for real-time fluids , 2006, SIGGRAPH '06.

[67]  G. Smith,et al.  Numerical Solution of Partial Differential Equations: Finite Difference Methods , 1978 .

[68]  Adrien Treuille,et al.  Keyframe control of smoke simulations , 2003, ACM Trans. Graph..

[69]  P. Werbos.  Backwards Differentiation in AD and Neural Nets: Past Links and New Opportunities , 2006 .