9 From the POD-Galerkin method to sparse manifold models

Reduced-order models are essential for the accurate and efficient prediction, estimation, and control of complex systems. This is especially true in fluid dynamics, where the fully resolved state space may easily contain millions or billions of degrees of freedom. Because these systems typically evolve on a low-dimensional attractor, model reduction is defined by two essential steps: 1) identify a good state space for the attractor, and 2) identify the dynamics on this attractor. The leading method for model reduction in fluids is Galerkin projection of the Navier-Stokes equations onto a linear subspace of modes obtained via proper orthogonal decomposition (POD). However, there are serious challenges in this approach, including truncation errors, stability issues, difficulty handling transients, and mode deformation with changing boundaries and operating conditions. Many of these challenges result from the choice of a linear POD subspace in which to represent the dynamics. In this chapter, we describe an alternative approach, feature-based manifold modeling (FeMM), in which the low-dimensional attractor and nonlinear dynamics are characterized from typical experimental data: time-resolved sensor data and optional non-time-resolved particle image velocimetry (PIV) snapshots. FeMM consists of three steps: First, the sensor signals are lifted to a dynamic feature space. Second, we identify a sparse human-interpretable nonlinear dynamical system for the feature state based on the sparse identification of nonlinear dynamics (SINDy). Third, if PIV snapshots are available, a local linear mapping from the feature state to the velocity field is performed to reconstruct the full-state of the system. We demonstrate this approach, and compare with POD-Galerkin modeling, on the incompressible two-dimensional flow around a circular cylinder. Best practices and perspectives for future research are also included, along with open-source code for this example.

[1]  B. R. Noack,et al.  Cluster-based network model of an incompressible mixing layer , 2020, 2001.02911.

[2]  I. Kovacic Nonlinear Oscillations , 2020 .

[3]  Wolfgang Schröder,et al.  Cluster‐based network model for drag reduction mechanisms of an actuated turbulent boundary layer , 2019, PAMM.

[4]  Alexandre Allauzen,et al.  Control of chaotic systems by deep reinforcement learning , 2019, Proceedings of the Royal Society A.

[5]  Frank Noé,et al.  Reactive SINDy: Discovering governing reactions from concentration data. , 2019, The Journal of chemical physics.

[6]  Victor Montagud-Camps Turbulence , 2019, Turbulent Heating and Anisotropy in the Solar Wind.

[7]  W. Hager,et al.  and s , 2019, Shallow Water Hydraulics.

[8]  Anthony T. Patera,et al.  A stabilized POD model for turbulent flows over a range of Reynolds numbers: Optimal parameter sampling and constrained projection , 2018, J. Comput. Phys..

[9]  Jean-Christophe Loiseau,et al.  Time-Stepping and Krylov Methods for Large-Scale Instability Problems , 2018, Computational Methods in Applied Sciences.

[10]  Petros Koumoutsakos,et al.  Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks , 2018, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[11]  E Kaiser,et al.  Sparse identification of nonlinear dynamics for model predictive control in the low-data limit , 2017, Proceedings of the Royal Society A.

[12]  Steven L. Brunton,et al.  Sparse reduced-order modelling: sensor-based dynamics to full-state estimation , 2017, Journal of Fluid Mechanics.

[13]  Steven L. Brunton,et al.  Constrained sparse Galerkin regression , 2016, Journal of Fluid Mechanics.

[14]  P. Meliga Harmonics generation and the mechanics of saturation in flow over an open cavity: a second-order self-consistent description , 2017, Journal of Fluid Mechanics.

[15]  Hayden Schaeffer,et al.  Sparse model selection via integral terms. , 2017, Physical review. E.

[16]  B. R. Noack,et al.  On the need of mode interpolation for data-driven Galerkin models of a transient flow around a sphere , 2017 .

[17]  Jan S. Hesthaven,et al.  Sparse identification of a predator-prey system from simulation data of a convection model , 2017 .

[18]  J N Kutz,et al.  Model selection for dynamical systems via sparse regression and information criteria , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[19]  Scott T. M. Dawson,et al.  Model Reduction for Flow Analysis and Control , 2017 .

[20]  Steven L. Brunton,et al.  Machine Learning Control – Taming Nonlinear Dynamics and Turbulence , 2016, Fluid Mechanics and Its Applications.

[21]  Steven L. Brunton,et al.  Data-driven discovery of partial differential equations , 2016, Science Advances.

[22]  Peter Jordan,et al.  Qualitative dynamics of wave packets in turbulent jets , 2016, 1608.06750.

[23]  Steven L. Brunton,et al.  Chaos as an intermittently forced linear system , 2016, Nature Communications.

[24]  Harbir Antil,et al.  Galerkin v. least-squares Petrov-Galerkin projection in nonlinear model reduction , 2015, J. Comput. Phys..

[25]  Clarence W. Rowley,et al.  Parameter-Varying Aerodynamics Models for Aggressive Pitching-Response Prediction , 2017 .

[26]  H. Schaeffer,et al.  Learning partial differential equations via data discovery and sparse optimization , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[27]  Steven L. Brunton,et al.  Dynamic mode decomposition - data-driven modeling of complex systems , 2016 .

[28]  Mariia Sorokina,et al.  Sparse identification for nonlinear optical communication systems: SINO method. , 2016, Optics express.

[29]  Jack J. McNamara,et al.  Model order reduction using sparse coding exemplified for the lid-driven cavity , 2016, Journal of Fluid Mechanics.

[30]  J. Templeton,et al.  Reynolds averaged turbulence modelling using deep neural networks with embedded invariance , 2016, Journal of Fluid Mechanics.

[31]  P. Meliga,et al.  A self-consistent formulation for the sensitivity analysis of finite-amplitude vortex shedding in the cylinder wake , 2016, Journal of Fluid Mechanics.

[32]  Lutz Lesshafft,et al.  Conditions for validity of mean flow stability analysis , 2016, Journal of Fluid Mechanics.

[33]  Steven L. Brunton,et al.  Inferring Biological Networks by Sparse Identification of Nonlinear Dynamics , 2016, IEEE Transactions on Molecular, Biological and Multi-Scale Communications.

[34]  Steven L. Brunton,et al.  Sparse Identification of Nonlinear Dynamics with Control (SINDYc) , 2016, 1605.06682.

[35]  Peter J. Schmid,et al.  Linear Closed-Loop Control of Fluid Instabilities and Noise-Induced Perturbations: A Review of Approaches and Tools , 2016 .

[36]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[37]  Hessam Babaee,et al.  A variational principle for the extraction of time-dependent modes associated with transient instabilities , 2015 .

[38]  B. R. Noack,et al.  Closed-Loop Turbulence Control: Progress and Challenges , 2015 .

[39]  Karthikeyan Duraisamy,et al.  Machine Learning Methods for Data-Driven Turbulence Modeling , 2015 .

[40]  Kunihiko Taira,et al.  Network-theoretic approach to sparsified discrete vortex dynamics , 2015, Journal of Fluid Mechanics.

[41]  M. Carini,et al.  Centre-manifold reduction of bifurcating flows , 2015, Journal of Fluid Mechanics.

[42]  Dwight Barkley,et al.  Prediction of frequencies in thermosolutal convection from mean flows. , 2015, Physical review. E, Statistical, nonlinear, and soft matter physics.

[43]  Clarence W. Rowley,et al.  A Data–Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition , 2014, Journal of Nonlinear Science.

[44]  Paul T. Boggs,et al.  Preserving Lagrangian Structure in Nonlinear Model Reduction with Application to Structural Dynamics , 2014, SIAM J. Sci. Comput..

[45]  B. R. Noack,et al.  On long-term boundedness of Galerkin models , 2013, Journal of Fluid Mechanics.

[46]  Gianluigi Rozza,et al.  Model Order Reduction: a survey , 2016 .

[47]  François Gallaire,et al.  Self-consistent mean flow description of the nonlinear saturation of the vortex shedding in the cylinder wake. , 2014, Physical review letters.

[48]  Jean-Christophe Loiseau Dynamics and global stability analysis of three-dimensional flows , 2014 .

[49]  N. Fabbiane,et al.  Adaptive and Model-Based Control Theory Applied to Convectively Unstable Flows , 2014, 1402.1746.

[50]  Steven L. Brunton,et al.  State-space model identification and feedback control of unsteady aerodynamic forces , 2014, 1401.1473.

[51]  Steven L. Brunton,et al.  On dynamic mode decomposition: Theory and applications , 2013, 1312.0041.

[52]  B. R. Noack,et al.  On the need for a nonlinear subscale turbulence term in POD models as exemplified for a high-Reynolds-number flow over an Ahmed body , 2013, Journal of Fluid Mechanics.

[53]  Bernd R. Noack,et al.  Cluster-based reduced-order modelling of a mixing layer , 2013, Journal of Fluid Mechanics.

[54]  Earl H. Dowell,et al.  Low-dimensional modelling of high-Reynolds-number shear flows incorporating constraints from the Navier–Stokes equation , 2013, Journal of Fluid Mechanics.

[55]  Gilead Tadmor,et al.  Reduced-Order Modelling for Flow Control , 2013 .

[56]  David R. Williams,et al.  Reduced-order unsteady aerodynamic models at low Reynolds numbers , 2013, Journal of Fluid Mechanics.

[57]  Bernhard Schölkopf,et al.  The Randomized Dependence Coefficient , 2013, NIPS.

[58]  I. Mezić,et al.  Analysis of Fluid Flows via Spectral Properties of the Koopman Operator , 2013 .

[59]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[61]  Weiwei Zhang,et al.  Efficient Method for Limit Cycle Flutter Analysis Based on Nonlinear Aerodynamic Reduced-Order Models , 2012 .

[62]  Traian Iliescu,et al.  Proper orthogonal decomposition closure models for turbulent flows: A numerical comparison , 2011, 1106.3585.

[63]  Gilead Tadmor,et al.  Reduced-order models for closed-loop wake control , 2011, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[64]  V. Theofilis Global Linear Instability , 2011 .

[65]  Alain Dervieux,et al.  Reduced-order modeling of transonic flows around an airfoil submitted to small deformations , 2011, J. Comput. Phys..

[66]  Bryan Glaz,et al.  Reduced-Order Nonlinear Unsteady Aerodynamic Modeling Using a Surrogate-Based Recurrence Framework , 2010 .

[67]  P. Meliga,et al.  Dynamics and Control of Global Instabilities in Open-Flows: A Linearized Approach , 2010 .

[68]  Gilead Tadmor,et al.  Mean field representation of the natural and actuated cylinder wake , 2010 .

[69]  P. Schmid,et al.  Dynamic mode decomposition of numerical and experimental data , 2008, Journal of Fluid Mechanics.

[70]  I. Mezić,et al.  Spectral analysis of nonlinear flows , 2009, Journal of Fluid Mechanics.

[71]  Dan S. Henningson,et al.  Matrix-Free Methods for the Stability and Control of Boundary Layers , 2009 .

[72]  Hod Lipson,et al.  Distilling Free-Form Natural Laws from Experimental Data , 2009, Science.

[73]  C. Rowley,et al.  Low-dimensional models of a temporally evolving free shear layer , 2009, Journal of Fluid Mechanics.

[74]  C. Farhat,et al.  Interpolation Method for Adapting Reduced-Order Models and Application to Aeroelasticity , 2008 .

[75]  Denis Sipp,et al.  Global stability of base and mean flows: a general approach and its applications to cylinder and open cavity flows , 2007, Journal of Fluid Mechanics.

[76]  Hod Lipson,et al.  Automated reverse engineering of nonlinear dynamical systems , 2007, Proceedings of the National Academy of Sciences.

[77]  P. Luchini,et al.  Structural sensitivity of the first instability of the cylinder wake , 2007, Journal of Fluid Mechanics.

[78]  Jing Wang,et al.  MLLE: Modified Locally Linear Embedding Using Multiple Weights , 2006, NIPS.

[79]  D. Barkley Linear analysis of the cylinder wake mean flow , 2006 .

[80]  O. Marxen,et al.  Steady solutions of the Navier-Stokes equations by selective frequency damping , 2006 .

[81]  Frank Thiele,et al.  Generalized Mean-Field Model for Flow Control Using a Continuous Mode Interpolation , 2006 .

[82]  B. R. Noack,et al.  Wake stabilization using POD Galerkin models with interpolated modes , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[83]  I. Mezić Spectral Properties of Dynamical Systems, Model Reduction and Decompositions , 2005 .

[84]  Bernd R. Noack,et al.  The need for a pressure-term representation in empirical Galerkin models of incompressible shear flows , 2005, Journal of Fluid Mechanics.

[85]  J. Chomaz,et al.  GLOBAL INSTABILITIES IN SPATIALLY DEVELOPING FLOWS: Non-Normality and Nonlinearity , 2005 .

[86]  Charles-Henri Bruneau,et al.  Low-order modelling of laminar flow regimes past a confined square cylinder , 2004, Journal of Fluid Mechanics.

[87]  G. Karniadakis,et al.  A spectral viscosity method for correcting the long-term behavior of POD models , 2004 .

[88]  B. R. Noack,et al.  A hierarchy of low-dimensional models for the transient and post-transient cylinder wake , 2003, Journal of Fluid Mechanics.

[89]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[90]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[91]  Michele Milano,et al.  Neural network modeling for near wall turbulent flow , 2002 .

[92]  Andrew J. Kurdila,et al.  Reduced Order Nonlinear Navier-Stokes Models for Synthetic Jets , 2002 .

[93]  Remi Manceau,et al.  Examination of large-scale structures in a turbulent plane mixing layer. Part 2. Dynamical systems model , 2001, Journal of Fluid Mechanics.

[94]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[95]  D. Rempfer,et al.  On Low-Dimensional Galerkin Models for Fluid Flow , 2000 .

[96]  Jean-Paul Bonnet,et al.  Examination of large-scale structures in a turbulent plane mixing layer. Part 1. Proper orthogonal decomposition , 1999, Journal of Fluid Mechanics.

[97]  Wr Graham,et al.  OPTIMAL CONTROL OF VORTEX SHEDDING USING LOW-ORDER MODELS. PART I-OPEN-LOOP MODEL DEVELOPMENT , 1999 .

[98]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[99]  P. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 1999 .

[100]  Bernhard Schölkopf,et al.  Kernel Principal Component Analysis , 1997, ICANN.

[101]  R. Goodman,et al.  Application of neural networks to turbulence control for drag reduction , 1997 .

[102]  R. Henderson,et al.  Three-dimensional Floquet stability analysis of the wake of a circular cylinder , 1996, Journal of Fluid Mechanics.

[103]  C. Williamson Vortex Dynamics in the Cylinder Wake , 1996 .

[104]  José Eduardo Wesfreid,et al.  On the spatial structure of global modes in wake flow , 1995 .

[105]  B. R. Noack,et al.  On the transition of the cylinder wake , 1995 .

[106]  Hermann F. Fasel,et al.  Dynamics of three-dimensional coherent structures in a flat-plate boundary layer , 1994, Journal of Fluid Mechanics.

[107]  Michael Schumm,et al.  Self-excited oscillations in the wake of two-dimensional bluff bodies and their control , 1994, Journal of Fluid Mechanics.

[108]  W. S. Edwards,et al.  Krylov methods for the incompressible Navier-Stokes equations , 1994 .

[109]  P. Holmes,et al.  The Proper Orthogonal Decomposition in the Analysis of Turbulent Flows , 1993 .

[110]  I. Kevrekidis,et al.  Low‐dimensional models for complex geometry flows: Application to grooved channels and circular cylinders , 1991 .

[111]  K. Sreenivasan,et al.  On the formation and suppression of vortex ‘shedding’ at low Reynolds numbers , 1990, Journal of Fluid Mechanics.

[112]  Nadine Aubry,et al.  The dynamics of coherent structures in the wall region of a turbulent boundary layer , 1988, Journal of Fluid Mechanics.

[113]  L. Sirovich Turbulence and the dynamics of coherent structures. I. Coherent structures , 1987 .

[114]  L. Sirovich Turbulence and the dynamics of coherent structures. II. Symmetries and transformations , 1987 .

[115]  K. Sreenivasan,et al.  HOPF BIFURCATION, LANDAU EQUATION, AND VORTEX SHEDDING BEHIND CIRCULAR CYLINDERS. , 1987 .

[116]  A. Zebib Stability of viscous flow past a circular cylinder , 1987 .

[117]  Jer-Nan Juang,et al.  An eigensystem realization algorithm for modal parameter identification and model reduction. [control systems design for large space structures] , 1985 .

[118]  M. Golubitsky,et al.  The Hopf Bifurcation , 1985 .

[119]  P. Holmes,et al.  Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields , 1983, Applied Mathematical Sciences.

[120]  W. Malkus,et al.  Outline of a theory of turbulent shear flow , 1956, Journal of Fluid Mechanics.

[121]  F. H. Adler Cybernetics, or Control and Communication in the Animal and the Machine. , 1949 .