An artificial neural network framework for reduced order modeling of transient flows

Abstract This paper proposes a supervised machine learning framework for the non-intrusive model order reduction of unsteady fluid flows to provide accurate predictions of non-stationary state variables when the control parameter values vary. Our approach utilizes a training process from full-order scale direct numerical simulation data projected on proper orthogonal decomposition (POD) modes to achieve an artificial neural network (ANN) model with reduced memory requirements. This data-driven ANN framework allows for a nonlinear time evolution of the modal coefficients without performing a Galerkin projection. Our POD-ANN framework can thus be considered an equation-free approach for latent space dynamics evolution of nonlinear transient systems and can be applied to a wide range of physical and engineering applications. Within this framework we introduce two architectures, namely sequential network (SN) and residual network (RN), to train the trajectory of modal coefficients. We perform a systematic analysis of the performance of the proposed reduced order modeling approaches on prediction of a nonlinear wave-propagation problem governed by the viscous Burgers equation, a simplified prototype setting for transient flows. We find that the POD-ANN-RN yields stable and accurate results for test problems assessed both within inside and outside of the database range and performs significantly better than the standard intrusive Galerkin projection model. Our results show that the proposed framework provides a non-intrusive alternative to the evolution of transient physics in a POD basis spanned space, and can be used as a robust predictive model order reduction tool for nonlinear dynamical systems.

[1]  John A. Burns,et al.  Using functional gains for effective sensor location in flow control: a reduced-order modelling approach , 2015, Journal of Fluid Mechanics.

[2]  Juan B. Valdés,et al.  NONLINEAR MODEL FOR DROUGHT FORECASTING BASED ON A CONJUNCTION OF WAVELET TRANSFORMS AND NEURAL NETWORKS , 2003 .

[3]  R. Freund Reduced-Order Modeling Techniques Based on Krylov Subspaces and Their Use in Circuit Simulation , 1999 .

[4]  J. Kutz,et al.  Compressive Sensing Based Machine Learning Strategy For Characterizing The Flow Around A Cylinder With Limited Pressure Measurements , 2013 .

[5]  Hugo F. S. Lui,et al.  Construction of reduced-order models for fluid flows using deep feedforward neural networks , 2019, Journal of Fluid Mechanics.

[6]  K. Kunisch,et al.  Optimal snapshot location for computing POD basis functions , 2010 .

[7]  Ahmed H. Elsheikh,et al.  DR-RNN: A deep residual recurrent neural network for model reduction , 2017, ArXiv.

[8]  Traian Iliescu,et al.  An evolve‐then‐filter regularized reduced order model for convection‐dominated flows , 2015, 1506.07555.

[9]  Eric Darve,et al.  The Neural Network Approach to Inverse Problems in Differential Equations , 2019, 1901.07758.

[10]  M. Yousuff Hussaini,et al.  Theoretical and computational fluid dynamics , 1989 .

[11]  D. A. Bistrian,et al.  Randomized dynamic mode decomposition for nonintrusive reduced order modelling , 2016, 1611.04884.

[12]  Brane Širok,et al.  Experimental Turbulent Field Modeling by Visualization and Neural Networks , 2004 .

[13]  S. Lele Compact finite difference schemes with spectral-like resolution , 1992 .

[14]  Mattan Kamon,et al.  Efficient Reduced-Order Modeling of Frequency-Dependent Coupling Inductances associated with 3-D Interconnect Structures , 1995, 32nd Design Automation Conference.

[15]  Zhu Wang,et al.  Artificial viscosity proper orthogonal decomposition , 2011, Math. Comput. Model..

[16]  M. Piggott,et al.  A POD reduced order unstructured mesh ocean modelling method for moderate Reynolds number flows , 2009 .

[17]  S. Michael Spottswood,et al.  A review of indirect/non-intrusive reduced order modeling of nonlinear geometric structures , 2013 .

[18]  Thomas A. Brenner,et al.  Acceleration techniques for reduced-order models based on proper orthogonal decomposition , 2008, J. Comput. Phys..

[19]  Kenneth Levenberg A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[20]  Bin Dong,et al.  Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations , 2017, ICML.

[21]  William E. Faller,et al.  Unsteady fluid mechanics applications of neural networks , 1995 .

[22]  Cédric Leblond,et al.  Optimal flow control using a POD-based reduced-order model , 2016 .

[23]  E. A. Gillies Low-dimensional control of the circular cylinder wake , 1998, Journal of Fluid Mechanics.

[24]  Nadine Aubry,et al.  The dynamics of coherent structures in the wall region of a turbulent boundary layer , 1988, Journal of Fluid Mechanics.

[25]  M. Samimy,et al.  Modeling of subsonic cavity flows by neural networks , 2004, Proceedings of the IEEE International Conference on Mechatronics, 2004. ICM '04..

[26]  Michel Loève,et al.  Probability Theory I , 1977 .

[27]  Eldad Haber,et al.  Stable architectures for deep neural networks , 2017, ArXiv.

[28]  Omer San,et al.  Machine learning closures for model order reduction of thermal fluids , 2018, Applied Mathematical Modelling.

[29]  Luigi Fortuna,et al.  Model Order Reduction Techniques with Applications in Electrical Engineering , 1992 .

[30]  C. Pain,et al.  Model identification of reduced order fluid dynamics systems using deep learning , 2018 .

[31]  Michael Y. Hu,et al.  Forecasting with artificial neural networks: The state of the art , 1997 .

[32]  Jean-Luc Aider,et al.  Closed-loop separation control using machine learning , 2014, Journal of Fluid Mechanics.

[33]  Laurent Cordier,et al.  New Regularization Method for Calibrated POD Reduced-Order Models , 2016 .

[34]  Michele Milano,et al.  Application of machine learning algorithms to flow modeling and optimization , 1999 .

[35]  Ming Su,et al.  Leapfrogging and synoptic Leapfrogging: A new optimization approach , 2012, Comput. Chem. Eng..

[36]  Ionel M. Navon,et al.  An improved algorithm for the shallow water equations model reduction: Dynamic Mode Decomposition vs POD , 2015 .

[37]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[38]  L. Sirovich Turbulence and the dynamics of coherent structures. I. Coherent structures , 1987 .

[39]  Stefan Volkwein,et al.  Galerkin Proper Orthogonal Decomposition Methods for a General Equation in Fluid Dynamics , 2002, SIAM J. Numer. Anal..

[40]  A. Hay,et al.  INTERVAL-BASED REDUCED-ORDER MODELS FOR UNSTEADY FLUID FLOW , 2007 .

[41]  Christopher C. Pain,et al.  A domain decomposition non-intrusive reduced order model for turbulent flows , 2019, Computers & Fluids.

[42]  Qian Wang,et al.  Non-intrusive reduced order modeling of unsteady flows using artificial neural networks with application to a combustion problem , 2019, J. Comput. Phys..

[43]  Petros Koumoutsakos,et al.  Data-assisted reduced-order modeling of extreme events in complex dynamical systems , 2018, PloS one.

[44]  C. Farhat,et al.  A low‐cost, goal‐oriented ‘compact proper orthogonal decomposition’ basis for model reduction of static systems , 2011 .

[45]  Ionel M. Navon,et al.  Non‐intrusive reduced order modelling with least squares fitting on a sparse grid , 2017 .

[46]  H. Hotelling Analysis of a complex of statistical variables into principal components. , 1933 .

[47]  Min Qi,et al.  Pricing and hedging derivative securities with neural networks: Bayesian regularization, early stopping, and bagging , 2001, IEEE Trans. Neural Networks.

[48]  Philip S. Beran,et al.  Reduced-order modeling - New approaches for computational physics , 2001 .

[49]  Satish Narayanan,et al.  Analysis of low dimensional dynamics of flow separation , 2000 .

[50]  B. R. Noack,et al.  Closed-Loop Turbulence Control: Progress and Challenges , 2015 .

[51]  Yin Wang,et al.  Sixth order compact scheme combined with multigrid method and extrapolation technique for 2D poisson equation , 2009, J. Comput. Phys..

[52]  P. Nair,et al.  Nonintrusive reduced‐order modeling of parametrized time‐dependent partial differential equations , 2013 .

[53]  Jose Flich,et al.  Flow Control , 2011, Encyclopedia of Parallel Computing.

[54]  J. Templeton Evaluation of machine learning algorithms for prediction of regions of high Reynolds averaged Navier Stokes uncertainty , 2015 .

[55]  A. Mohan,et al.  A Deep Learning based Approach to Reduced Order Modeling for Turbulent Flow Control using LSTM Neural Networks , 2018, 1804.09269.

[56]  J. Hesthaven,et al.  Non-intrusive reduced order modeling of nonlinear problems using neural networks , 2018, J. Comput. Phys..

[57]  J. Weller,et al.  Numerical methods for low‐order modeling of fluid flows based on POD , 2009 .

[58]  Charbel Farhat,et al.  Stabilization of projection‐based reduced‐order models , 2012 .

[59]  Akil C. Narayan,et al.  Practical error bounds for a non-intrusive bi-fidelity approach to parametric/stochastic model reduction , 2018, J. Comput. Phys..

[60]  Zhu Wang,et al.  Approximate Deconvolution Reduced Order Modeling , 2015, 1510.02726.

[61]  Omer San,et al.  Stabilized principal interval decomposition method for model reduction of nonlinear convective systems with moving shocks , 2018, Computational and Applied Mathematics.

[62]  Eric A Gillies Low-dimensional characterization and control of non-linear wake flows , 1995 .

[63]  Karen Willcox,et al.  A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems , 2015, SIAM Rev..

[64]  Omer San,et al.  Principal interval decomposition framework for POD reduced‐order modeling of convective Boussinesq flows , 2015 .

[65]  Karen Willcox,et al.  Goal-oriented, model-constrained optimization for reduction of large-scale systems , 2007, J. Comput. Phys..

[66]  J. Templeton,et al.  Reynolds averaged turbulence modelling using deep neural networks with embedded invariance , 2016, Journal of Fluid Mechanics.

[67]  Bernard Widrow,et al.  Neural networks: applications in industry, business and science , 1994, CACM.

[68]  Omer San,et al.  A neural network approach for the blind deconvolution of turbulent flows , 2017, Journal of Fluid Mechanics.

[69]  Jean-Antoine Désidéri,et al.  Stability Properties of POD–Galerkin Approximations for the Compressible Navier–Stokes Equations , 2000 .

[70]  Michele Milano,et al.  Neural network modeling for near wall turbulent flow , 2002 .

[71]  R.A. Sahan,et al.  Artificial neural network-based modeling and intelligent control of transitional flows , 1997, Proceedings of the 1997 IEEE International Conference on Control Applications.

[72]  Marco Debiasi,et al.  Control of Subsonic Cavity Flows by Neural Networks - Analytical Models and Experimental Validation , 2005 .

[73]  Billie F. Spencer,et al.  Modeling and Control of Magnetorheological Dampers for Seismic Response Reduction , 1996 .

[74]  R. Rico-Martinez,et al.  Low-dimensional models for active control of flow separation , 1999, Proceedings of the 1999 IEEE International Conference on Control Applications (Cat. No.99CH36328).

[75]  Jaijeet Roychowdhury,et al.  Reduced-order modeling of time-varying systems , 1999 .

[76]  Zhu Wang,et al.  Two-level discretizations of nonlinear closure models for proper orthogonal decomposition , 2011, J. Comput. Phys..

[77]  Julia Ling,et al.  Machine learning strategies for systems with invariance properties , 2016, J. Comput. Phys..

[78]  Bernd R. Noack,et al.  Identification strategies for model-based control , 2013 .

[79]  C. Pain,et al.  Non‐intrusive reduced‐order modelling of the Navier–Stokes equations based on RBF interpolation , 2015 .

[80]  Cheng Wang,et al.  Analysis of finite difference schemes for unsteady Navier-Stokes equations in vorticity formulation , 2002, Numerische Mathematik.

[81]  S. Ravindran A reduced-order approach for optimal control of fluids using proper orthogonal decomposition , 2000 .

[82]  Traian Iliescu,et al.  A stabilized proper orthogonal decomposition reduced-order model for large scale quasigeostrophic ocean circulation , 2014, Adv. Comput. Math..

[83]  B. R. Noack,et al.  On the need for a nonlinear subscale turbulence term in POD models as exemplified for a high-Reynolds-number flow over an Ahmed body , 2013, Journal of Fluid Mechanics.

[84]  Maria Vittoria Salvetti,et al.  Low-dimensional modelling of a confined three-dimensional wake flow , 2006, Journal of Fluid Mechanics.

[85]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[86]  E. A. Gillies Multiple sensor control of vortex shedding , 2001 .

[87]  Traian Iliescu,et al.  Energy balance and mass conservation in reduced order models of fluid flows , 2017, J. Comput. Phys..

[88]  Yuji Hattori,et al.  Searching for turbulence models by artificial neural network , 2016, 1607.01042.

[89]  Christian W. Dawson,et al.  An artificial neural network approach to rainfall-runoff modelling , 1998 .

[90]  Sirod Sirisup,et al.  On-line and Off-line POD Assisted Projective Integral for Non-linear Problems: A Case Study with Burgers-Equation , 2011 .

[91]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[92]  Martin T. Hagan,et al.  Neural network design , 1995 .

[93]  B. R. Noack Turbulence, Coherent Structures, Dynamical Systems and Symmetry , 2013 .

[94]  Ionel M. Navon,et al.  Non-intrusive reduced order modelling of the Navier-Stokes equations , 2015 .

[95]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[96]  P. Holmes,et al.  The Proper Orthogonal Decomposition in the Analysis of Turbulent Flows , 1993 .

[97]  Omer San,et al.  Extreme learning machine for reduced order modeling of turbulent geophysical flows. , 2018, Physical review. E.

[98]  Petros Boufounos,et al.  Sparse Sensing and DMD-Based Identification of Flow Regimes and Bifurcations in Complex Flows , 2015, SIAM J. Appl. Dyn. Syst..

[99]  Gilead Tadmor,et al.  Reduced-Order Modelling for Flow Control , 2013 .

[100]  Omer San,et al.  Neural network closures for nonlinear model order reduction , 2017, Adv. Comput. Math..

[101]  Scott T. M. Dawson,et al.  Model Reduction for Flow Analysis and Control , 2017 .

[102]  Claude Brezinski,et al.  Numerical Methods for Engineers and Scientists , 1992 .

[103]  G. Kerschen,et al.  The Method of Proper Orthogonal Decomposition for Dynamical Characterization and Order Reduction of Mechanical Systems: An Overview , 2005 .

[104]  Traian Iliescu,et al.  Proper orthogonal decomposition closure models for fluid flows: Burgers equation , 2013, 1308.3276.

[105]  K. Willcox,et al.  Data-driven operator inference for nonintrusive projection-based model reduction , 2016 .

[106]  M. Loève Probability theory : foundations, random sequences , 1955 .

[107]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[108]  Chi-Wang Shu,et al.  Total variation diminishing Runge-Kutta schemes , 1998, Math. Comput..

[109]  Traian Iliescu,et al.  Data-Driven Filtered Reduced Order Modeling of Fluid Flows , 2017, SIAM J. Sci. Comput..

[110]  D. Gottlieb,et al.  Stable and accurate boundary treatments for compact, high-order finite-difference schemes , 1993 .

[111]  R. Goodman,et al.  Application of neural networks to turbulence control for drag reduction , 1997 .

[112]  Brendan D. Tracey,et al.  Application of supervised learning to quantify uncertainties in turbulence and combustion modeling , 2013 .

[113]  J. Hesthaven,et al.  Greedy Nonintrusive Reduced Order Model for Fluid Dynamics , 2018, AIAA Journal.

[114]  Julia Kluge Applied And Computational Control Signals And Circuits , 2016 .

[115]  Traian Iliescu,et al.  Proper orthogonal decomposition closure models for turbulent flows: A numerical comparison , 2011, 1106.3585.

[116]  K. Duraisamy,et al.  Using field inversion to quantify functional errors in turbulence closures , 2016 .

[117]  Vassilios Theofilis,et al.  Modal Analysis of Fluid Flows: An Overview , 2017, 1702.01453.

[118]  H. Tran,et al.  Modeling and control of physical processes using proper orthogonal decomposition , 2001 .

[119]  Adrian Sandu,et al.  Efficient Construction of Local Parametric Reduced Order Models Using Machine Learning Techniques , 2015, ArXiv.

[120]  Zhu Wang,et al.  Numerical analysis of the Leray reduced order model , 2017, J. Comput. Appl. Math..