Construction of reduced-order models for fluid flows using deep feedforward neural networks

We present a numerical methodology for construction of reduced order models, ROMs, of fluid flows through the combination of flow modal decomposition and regression analysis. Spectral proper orthogonal decomposition, SPOD, is applied to reduce the dimensionality of the model and, at the same time, filter the POD temporal modes. The regression step is performed by a deep feedforward neural network, DNN, and the current framework is implemented in a context similar to the sparse identification of non-linear dynamics algorithm, SINDy. A discussion on the optimization of the DNN hyperparameters is provided for obtaining the best ROMs and an assessment of these models is presented for a canonical nonlinear oscillator and the compressible flow past a cylinder. Then, the method is tested on the reconstruction of a turbulent flow computed by a large eddy simulation of a plunging airfoil under dynamic stall. The reduced order model is able to capture the dynamics of the leading edge stall vortex and the subsequent trailing edge vortex. For the cases analyzed, the numerical framework allows the prediction of the flowfield beyond the training window using larger time increments than those employed by the full order model. We also demonstrate the robustness of the current ROMs constructed via deep feedforward neural networks through a comparison with sparse regression. The DNN approach is able to learn transient features of the flow and presents more accurate and stable long-term predictions compared to sparse regression.

[1]  Jinlong Wu,et al.  Physics-informed machine learning approach for reconstructing Reynolds stress modeling discrepancies based on DNS data , 2016, 1606.07987.

[2]  B. R. Noack,et al.  Optimal nonlinear eddy viscosity in Galerkin models of turbulent flows , 2014, Journal of Fluid Mechanics.

[3]  B. R. Noack,et al.  Closed-Loop Turbulence Control: Progress and Challenges , 2015 .

[4]  Melissa A. Green,et al.  Detection of Lagrangian coherent structures in three-dimensional turbulence , 2007, Journal of Fluid Mechanics.

[5]  T. Colonius,et al.  Spectral proper orthogonal decomposition and its relationship to dynamic mode decomposition and resolvent analysis , 2017, Journal of Fluid Mechanics.

[6]  J. Templeton Evaluation of machine learning algorithms for prediction of regions of high Reynolds averaged Navier Stokes uncertainty , 2015 .

[7]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[8]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[9]  Steven L. Brunton,et al.  On dynamic mode decomposition: Theory and applications , 2013, 1312.0041.

[10]  J. Nathan Kutz,et al.  Deep learning in fluid dynamics , 2017, Journal of Fluid Mechanics.

[11]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[12]  Karthik Duraisamy,et al.  Long-time predictive modeling of nonlinear dynamical systems using neural networks , 2018, Complex..

[13]  R. Murray,et al.  Model reduction for compressible flows using POD and Galerkin projection , 2004 .

[14]  S. Lele Compact finite difference schemes with spectral-like resolution , 1992 .

[15]  Danny C. Sorensen,et al.  Nonlinear Model Reduction via Discrete Empirical Interpolation , 2010, SIAM J. Sci. Comput..

[16]  P. Schmid,et al.  Dynamic mode decomposition of numerical and experimental data , 2008, Journal of Fluid Mechanics.

[17]  Earl H. Dowell,et al.  Minimal subspace rotation on the Stiefel manifold for stabilization and enhancement of projection-based reduced order models for the compressible Navier-Stokes equations , 2015, J. Comput. Phys..

[18]  Britton Jeffrey Olson Large-eddy simulation of multi-material mixing andover-expanded nozzle flow , 2012 .

[19]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[20]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[21]  Petros Koumoutsakos,et al.  Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks , 2018, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[24]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[25]  Miguel R. Visbal,et al.  Numerical Investigation of Deep Dynamic Stall of a Plunging Airfoil , 2010 .

[26]  Petros Koumoutsakos,et al.  Data-assisted reduced-order modeling of extreme events in complex dynamical systems , 2018, PloS one.

[27]  Karen Willcox,et al.  An Accelerated Greedy Missing Point Estimation Procedure , 2016, SIAM J. Sci. Comput..

[28]  Arthur Veldman,et al.  Proper orthogonal decomposition and low-dimensional models for driven cavity flows , 1998 .

[29]  Kunihiko Taira,et al.  Detecting Vortex Formation and Shedding in Cylinder Wakes Using Lagrangian Coherent Structures , 2017 .

[30]  R. Aris Vectors, Tensors and the Basic Equations of Fluid Mechanics , 1962 .

[31]  G. Haller Lagrangian Coherent Structures , 2015 .

[32]  Ricardo Dahab,et al.  Connecting the dots: Toward accountable machine-learning printer attribution methods , 2018, J. Vis. Commun. Image Represent..

[33]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[34]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[35]  Rashad Moarref,et al.  Low-dimensional representations of exact coherent states of the Navier-Stokes equations from the resolvent model of wall turbulence. , 2015, Physical review. E.

[36]  Wei Shyy,et al.  Shallow and deep dynamic stall for flapping low Reynolds number airfoils , 2009 .

[37]  Jer-Nan Juang,et al.  An eigensystem realization algorithm for modal parameter identification and model reduction. [control systems design for large space structures] , 1985 .

[38]  B. R. Noack,et al.  A hierarchy of low-dimensional models for the transient and post-transient cylinder wake , 2003, Journal of Fluid Mechanics.

[39]  L. Sirovich TURBULENCE AND THE DYNAMICS OF COHERENT STRUCTURES PART I : COHERENT STRUCTURES , 2016 .

[40]  B. J. McKeon,et al.  On the structure and origin of pressure fluctuations in wall turbulence: predictions based on the resolvent analysis , 2014, Journal of Fluid Mechanics.

[41]  C. Farhat,et al.  Efficient non‐linear model reduction via a least‐squares Petrov–Galerkin projection and compressive tensor approximations , 2011 .

[42]  S. Lele,et al.  Convective effects and the role of quadrupole sources for aerofoil aeroacoustics , 2012, Journal of Fluid Mechanics.

[43]  J. Templeton,et al.  Reynolds averaged turbulence modelling using deep neural networks with embedded invariance , 2016, Journal of Fluid Mechanics.

[44]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[45]  R. F. Warming,et al.  An Implicit Factored Scheme for the Compressible Navier-Stokes Equations , 1977 .

[46]  Michael Ol,et al.  Experimental Investigation of Pitching and Plunging Airfoils at Reynolds Number between 1x10^4 and 6x10^4 , 2009 .

[47]  B. R. Noack,et al.  On the need for a nonlinear subscale turbulence term in POD models as exemplified for a high-Reynolds-number flow over an Ahmed body , 2013, Journal of Fluid Mechanics.

[48]  Omer San,et al.  Extreme learning machine for reduced order modeling of turbulent geophysical flows. , 2018, Physical review. E.

[49]  K. Taira,et al.  High-Fidelity Simulation and Flow Control of a Plunging Airfoil under Deep Dynamic Stall , 2019, AIAA Scitech 2019 Forum.

[50]  Joel H. Ferziger,et al.  A robust high-order compact method for large eddy simulation , 2003 .

[51]  William Wolf,et al.  Identification of coherent structures in the flow past a NACA0012 airfoil via proper orthogonal decomposition , 2017 .

[52]  Steven L. Brunton,et al.  Data-driven discovery of partial differential equations , 2016, Science Advances.

[53]  Nando de Freitas,et al.  A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning , 2010, ArXiv.

[54]  Steven L. Brunton,et al.  Deep learning of dynamics and signal-noise decomposition with time-stepping constraints , 2018, J. Comput. Phys..

[55]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[56]  L. Sirovich Turbulence and the dynamics of coherent structures. I. Coherent structures , 1987 .

[57]  Heng Xiao,et al.  Data-Driven, Physics-Based Feature Extraction from Fluid Flow Fields using Convolutional Neural Networks , 2018, Communications in Computational Physics.

[58]  S. Lele,et al.  Investigation of Noise Generated by a DU96 Airfoil , 2012 .

[59]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[60]  R. Lewis,et al.  Low-storage, Explicit Runge-Kutta Schemes for the Compressible Navier-Stokes Equations , 2000 .

[61]  Vassilios Theofilis,et al.  Modal Analysis of Fluid Flows: An Overview , 2017, 1702.01453.

[62]  Christian Oliver Paschereit,et al.  Spectral proper orthogonal decomposition , 2015, Journal of Fluid Mechanics.