Neural-network learning of SPOD latent dynamics

We aim to reconstruct the latent space dynamics of high dimensional systems using model order reduction via the spectral proper orthogonal decomposition (SPOD). The proposed method is based on three fundamental steps: in the first, we compress the data from a high-dimensional representation to a lower dimensional one by constructing the SPOD latent space; in the second, we build the time-dependent coefficients by projecting the realizations (also referred to as snapshots) onto the reduced SPOD basis and we learn their evolution in time with the aid of recurrent neural networks; in the third, we reconstruct the high-dimensional data from the learnt lower-dimensional representation. The proposed method is demonstrated on two different test cases, namely, a compressible jet flow, and a geophysical problem known as the Madden-Julian Oscillation. An extensive comparison between SPOD and the equivalent POD-based counterpart is provided and differences between the two approaches are highlighted. The numerical results suggest that the proposed model is able to provide low rank predictions of complex statistically stationary data and to provide insights into the evolution of phenomena characterized by specific range of frequencies. The comparison between POD and SPOD surrogate strategies highlights the need for further work on the characterization of the error interplay between data reduction techniques and neural network forecasts.

[1]  Jan S. Hesthaven,et al.  Physics-informed machine learning for reduced-order modeling of nonlinear problems , 2021, J. Comput. Phys..

[2]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[3]  Gregory D. Hager,et al.  Temporal Convolutional Networks for Action Segmentation and Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[5]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[6]  H. Mori.  Transport, Collective Motion, and Brownian Motion , 1965 .

[7]  Petros,et al.  Generalized Stability Theory . Part I : Autonomous Operators , 2022 .

[8]  P. Holmes,et al.  The Proper Orthogonal Decomposition in the Analysis of Turbulent Flows , 1993 .

[9]  Gianmarco Mengaldo,et al.  PySPOD: A Python package for Spectral Proper Orthogonal Decomposition (SPOD) , 2021, J. Open Source Softw..

[10]  G. Balsamo,et al.  Spectral Empirical Orthogonal Function Analysis of Weather and Climate Data , 2019, Monthly Weather Review.

[11]  Prasanna Balaprakash,et al.  Time-series learning of latent-space dynamics for reduced-order model closure , 2019, Physica D: Nonlinear Phenomena.

[12]  R. Zwanzig.  Nonlinear generalized Langevin equations , 1973 .

[13]  J. Lumley.  Stochastic tools in turbulence , 1970 .

[14]  A. Föhrenbach,et al.  SIMPLE++ , 2000, OR Spectr..

[15]  J. Hesthaven,et al.  Non-intrusive reduced order modeling of nonlinear problems using neural networks , 2018, J. Comput. Phys..

[16]  Gianluigi Rozza,et al.  Data-Driven POD-Galerkin Reduced Order Model for Turbulent Flows , 2019, J. Comput. Phys..

[17]  L. Sirovich.  Turbulence and the dynamics of coherent structures. II. Symmetries and transformations , 1987 .

[18]  Sanjiva K. Lele,et al.  Modelling of jet noise: a perspective from large-eddy simulations , 2019, Philosophical Transactions of the Royal Society A.

[19]  K. Huth.  Transport , 2015, Canadian Medical Association Journal.

[20]  P. Alam.  ‘A’ , 2021, Composites Engineering: An A–Z Guide.

[21]  Alex Sherstinsky,et al.  Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network , 2018, Physica D: Nonlinear Phenomena.

[22]  S. Lele,et al.  Broadband reconstruction of inhomogeneous turbulence using spectral proper orthogonal decomposition and Gabor modes , 2020, Journal of Fluid Mechanics.

[23]  Prashant D. Sardeshmukh,et al.  Reconciling Non-Gaussian Climate Statistics with Linear Dynamics , 2009 .

[24]  Koji Fukagata,et al.  CNN-LSTM based reduced order modeling of two-dimensional unsteady flows around a circular cylinder at different Reynolds numbers , 2020, Fluid Dynamics Research.

[25]  O. Schmidt,et al.  Frequency–time analysis, low-rank reconstruction and denoising of turbulent flows using SPOD , 2020, Journal of Fluid Mechanics.

[26]  B. Hamzi,et al.  Simple, low-cost and accurate data-driven geophysical forecasting with learned kernels , 2021, Proceedings of the Royal Society A.

[27]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[28]  G. North,et al.  Empirical Orthogonal Functions: The Medium is the Message , 2009 .

[29]  Qian Wang,et al.  Recurrent neural network closure of parametric POD-Galerkin reduced-order models based on the Mori-Zwanzig formalism , 2020, J. Comput. Phys..

[30]  T. Colonius,et al.  Spectral proper orthogonal decomposition and its relationship to dynamic mode decomposition and resolvent analysis , 2017, Journal of Fluid Mechanics.

[31]  A. Mohan,et al.  A Deep Learning based Approach to Reduced Order Modeling for Turbulent Flow Control using LSTM Neural Networks , 2018, 1804.09269.

[32]  Gary P. Morriss,et al.  Statistical Mechanics of Nonequilibrium Liquids , 2008 .

[33]  Chidong Zhang,et al.  Madden‐Julian Oscillation , 2005 .

[34]  Francisco J. Gonzalez,et al.  Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems , 2018, ArXiv.

[35]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[36]  E Weinan,et al.  Model Reduction with Memory and the Machine Learning of Dynamical Systems , 2018, Communications in Computational Physics.

[37]  R. Maulik,et al.  Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders , 2020, 2002.00470.