Deep Conservation: A Latent-Dynamics Model for Exact Satisfaction of Physical Conservation Laws

This work proposes an approach for latent dynamics learning that exactly enforces physical conservation laws. The method comprises two steps. First, we compute a low-dimensional embedding of the high-dimensional dynamical-system state using deep convolutional autoencoders. This defines a low-dimensional nonlinear manifold on which the state is subsequently enforced to evolve. Second, we define a latent dynamics model that associates with a constrained optimization problem. Specifically, the objective function is defined as the sum of squares of conservation-law violations over control volumes in a finite-volume discretization of the problem; nonlinear equality constraints explicitly enforce conservation over prescribed subdomains of the problem. The resulting dynamics model-which can be considered as a projection-based reduced-order model-ensures that the time-evolution of the latent state exactly satisfies conservation laws over the prescribed subdomains. In contrast to existing methods for latent dynamics learning, this is the only method that both employs a nonlinear embedding and computes dynamics for the latent state that guarantee the satisfaction of prescribed physical properties. Numerical experiments on a benchmark advection problem illustrate the method's ability to significantly reduce the dimensionality while enforcing physical conservation.

[1]  C. Hirsch Numerical computation of internal and external flows , 1988 .

[2]  Garrison W. Cottrell,et al.  Non-Linear Dimensionality Reduction , 1992, NIPS.

[3]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[4]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[5]  R. Eymard,et al.  Finite Volume Methods , 2019, Computational Methods for Fluid Dynamics.

[6]  R. LeVeque Finite Volume Methods for Hyperbolic Problems: Characteristics and Riemann Problems for Linear Hyperbolic Equations , 2002 .

[7]  RewieÅ ski,et al.  A trajectory piecewise-linear approach to model order reduction of nonlinear dynamical systems , 2003 .

[8]  Michal Rewienski,et al.  A trajectory piecewise-linear approach to model order reduction of nonlinear dynamical systems , 2003 .

[9]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[10]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[11]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[12]  C. Farhat,et al.  Efficient non‐linear model reduction via a least‐squares Petrov–Galerkin projection and compressive tensor approximations , 2011 .

[13]  B. R. Noack Turbulence, Coherent Structures, Dynamical Systems and Symmetry , 2013 .

[14]  Charbel Farhat,et al.  Progressive construction of a parametric reduced‐order model for PDE‐constrained optimization , 2014, ArXiv.

[15]  Klaus Obermayer,et al.  Autonomous Learning of State Representations for Control: An Emerging Field Aims to Autonomously Learn State Representations for Reinforcement Learning Agents from Their Real-World Sensor Observations , 2015, KI - Künstliche Intelligenz.

[16]  Karen Willcox,et al.  A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems , 2015, SIAM Rev..

[17]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[18]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[19]  Mario Ohlberger,et al.  Reduced Basis Methods: Success, Limitations and Future Challenges , 2015, 1511.02021.

[20]  Martin A. Riedmiller,et al.  Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images , 2015, NIPS.

[21]  Yann LeCun,et al.  Learning to Linearize Under Uncertainty , 2015, NIPS.

[22]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[23]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[24]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[25]  Harbir Antil,et al.  Galerkin v. least-squares Petrov-Galerkin projection in nonlinear model reduction , 2015, J. Comput. Phys..

[26]  Naoya Takeishi,et al.  Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition , 2017, NIPS.

[27]  Maximilian Karl,et al.  Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data , 2016, ICLR.

[28]  Ali Ghodsi,et al.  Robust Locally-Linear Controllable Embedding , 2017, AISTATS.

[29]  Francisco J. Gonzalez,et al.  Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems , 2018, ArXiv.

[30]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[31]  Mykel J. Kochenderfer,et al.  Deep Dynamical Modeling and Control of Unsteady Fluid Flows , 2018, NeurIPS.

[32]  Youngsoo Choi,et al.  Conservative model reduction for finite-volume models , 2017, J. Comput. Phys..

[33]  Steven L. Brunton,et al.  Deep learning for universal linear embeddings of nonlinear dynamics , 2017, Nature Communications.

[34]  Jorge Nocedal,et al.  Optimization Methods for Large-Scale Machine Learning , 2016, SIAM Rev..

[35]  David Filliat,et al.  State Representation Learning for Control: An Overview , 2018, Neural Networks.

[36]  Clarence W. Rowley,et al.  Linearly-Recurrent Autoencoder Networks for Learning Dynamics , 2017, SIAM J. Appl. Dyn. Syst..

[37]  Pierre Gentine,et al.  Achieving Conservation of Energy in Neural Network Emulators for Climate Modeling , 2019, ArXiv.

[38]  David I. W. Levin,et al.  Latent‐space Dynamics for Reduced Deformable Simulation , 2019, Comput. Graph. Forum.

[39]  Michael Chertkov,et al.  Turbulence forecasting via Neural ODE. , 2019, 1911.05180.

[40]  Jason Yosinski,et al.  Hamiltonian Neural Networks , 2019, NeurIPS.

[41]  Victor Montagud-Camps Turbulence , 2019, Turbulent Heating and Anisotropy in the Solar Wind.

[42]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[43]  Nils Thürey,et al.  Latent Space Physics: Towards Learning the Temporal Evolution of Fluid Flow , 2018, Comput. Graph. Forum.

[44]  Miles Cranmer,et al.  Lagrangian Neural Networks , 2020, ICLR 2020.

[45]  Danilo Jimenez Rezende,et al.  Hamiltonian Generative Networks , 2019, ICLR.

[46]  Prasanna Balaprakash,et al.  Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders , 2020, Physics of Fluids.

[47]  Kookjin Lee,et al.  Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders , 2018, J. Comput. Phys..

[48]  L. Bottou,et al.  Symplectic Recurrent Neural Networks , 2019, ICLR.