Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders

A common strategy for the dimensionality reduction of nonlinear partial differential equations relies on the use of the proper orthogonal decomposition (POD) to identify a reduced subspace and the Galerkin projection for evolving dynamics in this reduced space. However, advection-dominated PDEs are represented poorly by this methodology since the process of truncation discards important interactions between higher-order modes during time evolution. In this study, we demonstrate that an encoding using convolutional autoencoders (CAEs) followed by a reduced-space time evolution by recurrent neural networks overcomes this limitation effectively. We demonstrate that a truncated system of only two latent-space dimensions can reproduce a sharp advecting shock profile for the viscous Burgers equation with very low viscosities, and a twelve-dimensional latent space can recreate the evolution of the inviscid shallow water equations. Additionally, the proposed framework is extended to a parametric reduced-order model by directly embedding parametric information into the latent space to detect trends in system evolution. Our results show that these advection-dominated systems are more amenable to low-dimensional encoding and time evolution by a CAE and recurrent neural network combination than the POD Galerkin technique.

[1]  Traian Iliescu,et al.  Proper orthogonal decomposition closure models for turbulent flows: A numerical comparison , 2011, 1106.3585.

[2]  Petros Koumoutsakos,et al.  Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks , 2018, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[3]  Qian Wang,et al.  Non-intrusive reduced order modeling of unsteady flows using artificial neural networks with application to a combustion problem , 2019, J. Comput. Phys..

[4]  Omer San,et al.  Extreme learning machine for reduced order modeling of turbulent geophysical flows. , 2018, Physical review. E.

[5]  Gilead Tadmor,et al.  Reduced-Order Modelling for Flow Control , 2013 .

[6]  Omer San,et al.  Neural network closures for nonlinear model order reduction , 2017, Adv. Comput. Math..

[7]  Francisco J. Gonzalez,et al.  Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems , 2018, ArXiv.

[8]  Traian Iliescu,et al.  A long short-term memory embedding for hybrid uplifted reduced order models , 2019, Physica A: Statistical Mechanics and its Applications.

[9]  Omer San,et al.  An artificial neural network framework for reduced order modeling of transient flows , 2018, Commun. Nonlinear Sci. Numer. Simul..

[10]  Youngsoo Choi,et al.  Space-time least-squares Petrov-Galerkin projection for nonlinear model reduction , 2017, SIAM J. Sci. Comput..

[11]  Traian Iliescu,et al.  An evolve‐then‐filter regularized reduced order model for convection‐dominated flows , 2015, 1506.07555.

[12]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[13]  Eyad H. Abed,et al.  Local modal participation analysis of nonlinear systems using Poincaré linearization , 2019, Nonlinear Dynamics.

[14]  Karen Willcox,et al.  Nonlinear Model Order Reduction via Lifting Transformations and Proper Orthogonal Decomposition , 2018, AIAA Journal.

[15]  Virginia Kalb,et al.  An intrinsic stabilization scheme for proper orthogonal decomposition based low-dimensional models , 2007 .

[16]  Harbir Antil,et al.  Galerkin v. least-squares Petrov-Galerkin projection in nonlinear model reduction , 2015, J. Comput. Phys..

[17]  Omer San,et al.  A dynamic closure modeling framework for model order reduction of geophysical flows , 2019, Physics of Fluids.

[18]  C. Farhat,et al.  Efficient non‐linear model reduction via a least‐squares Petrov–Galerkin projection and compressive tensor approximations , 2011 .

[19]  Francisco Javier Gonzalez,et al.  Learning low-dimensional feature dynamics using convolutional recurrent autoencoders , 2018 .

[20]  Michael Dellnitz,et al.  Multiobjective Optimal Control Methods for the Navier-Stokes Equations Using Reduced Order Modeling , 2018, Acta Applicandae Mathematicae.

[21]  Christopher C. Pain,et al.  Optimal reduced space for Variational Data Assimilation , 2019, J. Comput. Phys..

[22]  Matthew F. Barone,et al.  On the stability and convergence of a Galerkin reduced order model (ROM) of compressible flow with solid wall and far‐field boundary treatment , 2010 .

[23]  Gianluigi Rozza,et al.  Supremizer stabilization of POD–Galerkin approximation of parametrized steady incompressible Navier–Stokes equations , 2015 .

[24]  Omer San,et al.  Principal interval decomposition framework for POD reduced‐order modeling of convective Boussinesq flows , 2015 .

[25]  Annalisa Quaini,et al.  A localized reduced-order modeling approach for PDEs with bifurcating solutions , 2018, Computer Methods in Applied Mechanics and Engineering.

[26]  Quoc V. Le,et al.  Searching for Activation Functions , 2018, arXiv.

[27]  P. Holmes,et al.  The Proper Orthogonal Decomposition in the Analysis of Turbulent Flows , 1993 .

[28]  Steven L. Brunton,et al.  Deep learning for universal linear embeddings of nonlinear dynamics , 2017, Nature Communications.

[29]  Steven L. Brunton,et al.  Dynamic Mode Decomposition with Control , 2014, SIAM J. Appl. Dyn. Syst..

[30]  S. Osher,et al.  Weighted essentially non-oscillatory schemes , 1994 .

[31]  Qian Wang,et al.  Recurrent neural network closure of parametric POD-Galerkin reduced-order models based on the Mori-Zwanzig formalism , 2020, J. Comput. Phys..

[32]  Matthew J. Zahr,et al.  An Efficient, Globally Convergent Method for Optimization Under Uncertainty Using Adaptive Model Reduction and Sparse Grids , 2019, SIAM/ASA J. Uncertain. Quantification.

[33]  A. Mohan,et al.  A Deep Learning based Approach to Reduced Order Modeling for Turbulent Flow Control using LSTM Neural Networks , 2018, 1804.09269.

[34]  Michael W. Mahoney,et al.  Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction , 2019, ArXiv.

[35]  Juan Du,et al.  Non-linear Petrov-Galerkin methods for reduced order hyperbolic equations and discontinuous finite element methods , 2013, J. Comput. Phys..

[36]  Scott T. M. Dawson,et al.  Model Reduction for Flow Analysis and Control , 2017 .

[37]  F. Krogh,et al.  Solving Ordinary Differential Equations , 2019, Programming for Computations - Python.

[38]  Benjamin Peherstorfer,et al.  Optimal Model Management for Multifidelity Monte Carlo Estimation , 2016, SIAM J. Sci. Comput..

[39]  R. Maulik,et al.  Resolution and Energy Dissipation Characteristics of Implicit LES and Explicit Filtering Models for Compressible Turbulence , 2017 .

[40]  Ionel M. Navon,et al.  Non-linear Petrov-Galerkin methods for reduced order modelling of the Navier-Stokes equations using a mixed finite element pair , 2013 .

[41]  Benjamin Peherstorfer,et al.  Projection-based model reduction: Formulations for physics-based machine learning , 2019, Computers & Fluids.

[42]  M. Mohebujjaman,et al.  Physically constrained data‐driven correction for reduced‐order modeling of fluid flows , 2018, International Journal for Numerical Methods in Fluids.

[43]  Prasanna Balaprakash,et al.  Time-series learning of latent-space dynamics for reduced-order model closure , 2019, Physica D: Nonlinear Phenomena.

[44]  Andrew J Majda,et al.  Statistically accurate low-order models for uncertainty quantification in turbulent dynamical systems , 2013, Proceedings of the National Academy of Sciences.

[45]  Traian Iliescu,et al.  Data-Driven Filtered Reduced Order Modeling of Fluid Flows , 2017, SIAM J. Sci. Comput..

[46]  Kookjin Lee,et al.  Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders , 2018, J. Comput. Phys..

[47]  Leonidas J. Guibas,et al.  A Point-Cloud Deep Learning Framework for Prediction of Fluid Flow Fields on Irregular Geometries , 2020, ArXiv.

[48]  Karthik Duraisamy,et al.  Modal Analysis of Fluid Flows: Applications and Outlook , 2019, AIAA Journal.

[49]  Karthik Duraisamy,et al.  Multi-level Convolutional Autoencoder Networks for Parametric Prediction of Spatio-temporal Dynamics , 2019, Computer Methods in Applied Mechanics and Engineering.

[50]  A. Mohan,et al.  Compressed Convolutional LSTM: An Efficient Deep Learning framework to Model High Fidelity 3D Turbulence , 2019, 1903.00033.

[51]  D. D. Kosambi.  Statistics in Function Space , 2016 .

[52]  Milan Korda,et al.  Data-driven spectral analysis of the Koopman operator , 2017, Applied and Computational Harmonic Analysis.

[53]  Paul J. Atzberger,et al.  GMLS-Nets: A framework for learning from unstructured data , 2019, ArXiv.