Machine Learning the Tangent Space of Dynamical Instabilities from Data

For a large class of dynamical systems, the optimally time-dependent (OTD) modes, a set of deformable orthonormal tangent vectors that track directions of instabilities along any trajectory, are known to depend "pointwise" on the state of the system on the attractor but not on the history of the trajectory. We leverage the power of neural networks to learn this "pointwise" mapping from the phase space to OTD space directly from data. The result of the learning process is a cartography of directions associated with strongest instabilities in the phase space. Implications for data-driven prediction and control of dynamical instabilities are discussed.

[1]  Asger M. Haugaard.  Predicting the bounds of large chaotic systems using low-dimensional manifolds , 2017, PloS one.

[2]  A. B. Potapov,et al.  On the concept of stationary Lyapunov basis , 1998 .

[3]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[4]  R. Samelson,et al.  An efficient method for recovering Lyapunov vectors from singular vectors , 2007 .

[5]  V. I. Oseledec.  A multiplicative ergodic theorem: Lyapunov characteristic num-bers for dynamical systems , 1968 .

[6]  Saso Dzeroski,et al.  Declarative Bias in Equation Discovery , 1997, ICML.

[7]  Antoine Blanchard,et al.  Stabilization of unsteady flows by reduced-order control with optimally time-dependent modes , 2019, Physical Review Fluids.

[8]  J. Yosinski,et al.  Time-series Extreme Event Forecasting with Neural Networks at Uber , 2017 .

[9]  George E. Karniadakis,et al.  Hidden physics models: Machine learning of nonlinear partial differential equations , 2017, J. Comput. Phys..

[10]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[11]  Bin Dong,et al.  PDE-Net: Learning PDEs from Data , 2017, ICML.

[12]  H. Babaee,et al.  A minimization principle for the description of modes associated with finite-time instabilities , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[13]  Saso Dzeroski,et al.  Integrating Knowledge-Driven and Data-Driven Approaches to Modeling , 2006, EnviroInfo.

[14]  张振跃,et al.  Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment , 2004 .

[15]  Bernd R. Noack,et al.  A global stability analysis of the steady and periodic cylinder wake , 1994, Journal of Fluid Mechanics.

[16]  Neil Fenichel.  Geometric singular perturbation theory for ordinary differential equations , 1979 .

[17]  Tina Toni,et al.  Designing attractive models via automated identification of chaotic and oscillatory dynamical regimes , 2011, Nature communications.

[18]  H. Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[19]  Jaideep Pathak,et al.  Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data. , 2017, Chaos.

[20]  Petros Koumoutsakos,et al.  Data-assisted reduced-order modeling of extreme events in complex dynamical systems , 2018, PloS one.

[21]  Paris Perdikaris,et al.  Inferring solutions of differential equations using noisy multi-fidelity data , 2016, J. Comput. Phys..

[22]  T. Sapsis,et al.  Reduced-order description of transient instabilities and computation of finite-time Lyapunov exponents. , 2017, Chaos.

[23]  Karthik Duraisamy,et al.  Turbulence Modeling in the Age of Data , 2018, Annual Review of Fluid Mechanics.

[24]  Raman Sujith,et al.  Thermoacoustic instability in a Rijke tube: non-normality and nonlinearity , 2007 .

[25]  Anne E. Trefethen,et al.  Hydrodynamic Stability Without Eigenvalues , 1993, Science.

[26]  D. Ruelle,et al.  Ergodic theory of chaos and strange attractors , 1985 .

[27]  Eckmann,et al.  Liapunov exponents from time series. , 1986, Physical review. A, General physics.

[28]  M. Mackey,et al.  Probabilistic properties of deterministic systems , 1985, Acta Applicandae Mathematicae.

[29]  L. Sirovich.  Turbulence and the dynamics of coherent structures. I. Coherent structures , 1987 .

[30]  Steven A. Orszag,et al.  Stability and Lyapunov stability of dynamical systems: A differential approach and a numerical method , 1987 .

[31]  George Em Karniadakis,et al.  Inferring solutions of dierential equations using noisy multi-delity data , 2016 .

[32]  Justin A. Sirignano,et al.  DGM: A deep learning algorithm for solving partial differential equations , 2017, J. Comput. Phys..

[33]  Charbel Farhat,et al.  Nonlinear model order reduction based on local reduced‐order bases , 2012 .

[34]  George Em Karniadakis,et al.  Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks , 2019, SIAM J. Sci. Comput..

[35]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[36]  Jorge Nocedal,et al.  Optimization Methods for Large-Scale Machine Learning , 2016, SIAM Rev..

[37]  H. Kantz.  A robust method to estimate the maximal Lyapunov exponent of a time series , 1994 .

[38]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[39]  Quoc V. Le,et al.  Swish: a Self-Gated Activation Function , 2017, 1710.05941.

[40]  An efficient method for recovering Lyapunov vectors from singular vectors , 2007 .

[41]  P. Schmid.  Nonmodal Stability Theory , 2007 .

[42]  Petros Koumoutsakos,et al.  Machine Learning for Fluid Mechanics , 2019, Annual Review of Fluid Mechanics.

[43]  Michelle Girvan,et al.  Forecasting of Spatio-temporal Chaotic Dynamics with Recurrent Neural Networks: a comparative study of Reservoir Computing and Backpropagation Algorithms , 2019, ArXiv.

[44]  P. Holmes,et al.  Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields , 1983, Applied Mathematical Sciences.

[45]  Mohammad Farazmand,et al.  Dynamical indicators for the prediction of bursting phenomena in high-dimensional systems. , 2016, Physical review. E.

[46]  Ferdinand Verhulst,et al.  A Mechanism for Atmospheric Regime Behavior , 2004 .

[47]  Maziar Raissi,et al.  Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations , 2018, J. Mach. Learn. Res..

[48]  Prashant D. Sardeshmukh,et al.  The Optimal Growth of Tropical Sea Surface Temperature Anomalies , 1995 .

[49]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[50]  Richard J. Lipton,et al.  Multidimensional Searching Problems , 1976, SIAM J. Comput..

[51]  D. Solli,et al.  Recent progress in investigating optical rogue waves , 2013 .

[52]  B. R. Noack,et al.  A hierarchy of low-dimensional models for the transient and post-transient cylinder wake , 2003, Journal of Fluid Mechanics.

[53]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[54]  Antoine Blanchard,et al.  Analytical Description of Optimally Time-Dependent Modes for Reduced-Order Modeling of Transient Instabilities , 2019, SIAM J. Appl. Dyn. Syst..

[55]  Bernd R. Noack,et al.  Cluster-based reduced-order modelling of a mixing layer , 2013, Journal of Fluid Mechanics.

[56]  Quoc V. Le,et al.  Searching for Activation Functions , 2018, arXiv.

[57]  M. Rosenstein,et al.  A practical method for calculating largest Lyapunov exponents from small data sets , 1993 .

[58]  Zhiping Mao,et al.  DeepXDE: A Deep Learning Library for Solving Differential Equations , 2019, AAAI Spring Symposium: MLPS.

[59]  Sawada,et al.  Measurement of the Lyapunov spectrum from a chaotic time series. , 1985, Physical review letters.

[60]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[61]  T. Sapsis,et al.  Control of linear instabilities by dynamically consistent order reduction on optimally time-dependent modes , 2019, Nonlinear Dynamics.

[62]  C. Farhat,et al.  Interpolation Method for Adapting Reduced-Order Models and Application to Aeroelasticity , 2008 .

[63]  J. Chomaz,et al.  GLOBAL INSTABILITIES IN SPATIALLY DEVELOPING FLOWS: Non-Normality and Nonlinearity , 2005 .