Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition

Spectral decomposition of the Koopman operator is attracting attention as a tool for the analysis of nonlinear dynamical systems. Dynamic mode decomposition is a popular numerical algorithm for Koopman spectral analysis; however, we often need to prepare nonlinear observables manually according to the underlying dynamics, which is not always possible since we may not have any a priori knowledge about them. In this paper, we propose a fully data-driven method for Koopman spectral analysis based on the principle of learning Koopman invariant subspaces from observed data. To this end, we propose minimization of the residual sum of squares of linear least-squares regression to estimate a set of functions that transforms data into a form in which the linear regression fits well. We introduce an implementation with neural networks and evaluate performance empirically using nonlinear dynamical systems and applications.

[1]  Naoya Takeishi,et al.  Subspace dynamic mode decomposition for stochastic Koopman analysis. , 2017, Physical review. E.

[2]  Diederik P. Kingma,et al.  Stochastic Gradient VB and the Variational Auto-Encoder , 2013 .

[3]  Uri Shalit,et al.  Structured Inference Networks for Nonlinear State Space Models , 2016, AAAI.

[4]  Soumya Kundu,et al.  Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems , 2017, 2019 American Control Conference (ACC).

[5]  S. P. Garcia,et al.  Multivariate phase space reconstruction by nearest neighbor embedding with different time delays. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[6]  Steven L. Brunton,et al.  Chaos as an intermittently forced linear system , 2016, Nature Communications.

[7]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[8]  Igor Mezic,et al.  Ergodic Theory, Dynamic Mode Decomposition, and Computation of Spectral Properties of the Koopman Operator , 2016, SIAM J. Appl. Dyn. Syst..

[9]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[10]  Alexandre Mauroy,et al.  Linear identification of nonlinear systems: A lifting technique based on the Koopman operator , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[11]  Lawrence F. Shampine,et al.  The MATLAB ODE Suite , 1997, SIAM J. Sci. Comput..

[12]  Vladimir Rakočević,et al.  On continuity of the Moore-Penrose and Drazin inverses. , 1997 .

[13]  Nigel Collier,et al.  Change-Point Detection in Time-Series Data by Relative Density-Ratio Estimation , 2012, Neural Networks.

[14]  Steven L. Brunton,et al.  Dynamic mode decomposition - data-driven modeling of complex systems , 2016 .

[15]  Yoshua Bengio,et al.  A Recurrent Latent Variable Model for Sequential Data , 2015, NIPS.

[16]  M. Hirsch,et al.  Differential Equations, Dynamical Systems, and an Introduction to Chaos , 2003 .

[17]  Clarence W. Rowley,et al.  Linearly-Recurrent Autoencoder Networks for Learning Dynamics , 2017, SIAM J. Appl. Dyn. Syst..

[18]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[19]  B. O. Koopman,et al.  Hamiltonian Systems and Transformation in Hilbert Space. , 1931, Proceedings of the National Academy of Sciences of the United States of America.

[20]  Steven L. Brunton,et al.  On dynamic mode decomposition: Theory and applications , 2013, 1312.0041.

[21]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[22]  Prateek Jain,et al.  Online and Stochastic Gradient Methods for Non-decomposable Loss Functions , 2014, NIPS.

[23]  Ryan P. Adams,et al.  Composing graphical models with neural networks for structured representations and fast inference , 2016, NIPS.

[24]  Heni Ben Amor,et al.  Estimation of perturbations in robotic behavior using dynamic mode decomposition , 2015, Adv. Robotics.

[25]  Clarence W. Rowley,et al.  A Data–Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition , 2014, Journal of Nonlinear Science.

[26]  I. Mezić.  Spectral Properties of Dynamical Systems, Model Reduction and Decompositions , 2005 .

[27]  Alexander J. Smola,et al.  Kernel methods and the exponential family , 2006, ESANN.

[28]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[29]  Steven L. Brunton,et al.  Multiresolution Dynamic Mode Decomposition , 2015, SIAM J. Appl. Dyn. Syst..

[30]  John P. Cunningham,et al.  Linear dynamical neural population models through nonlinear embeddings , 2016, NIPS.

[31]  Hao Wu,et al.  VAMPnets for deep learning of molecular kinetics , 2017, Nature Communications.

[32]  Yoshinobu Kawahara,et al.  Dynamic Mode Decomposition with Reproducing Kernels for Koopman Spectral Analysis , 2016, NIPS.

[33]  E. Lorenz.  Deterministic nonperiodic flow , 1963 .

[34]  Gary Froyland,et al.  A Computational Method to Extract Macroscopic Variables and Their Dynamics in Multiscale Systems , 2013, SIAM J. Appl. Dyn. Syst..

[35]  Zoubin Ghahramani,et al.  Learning Nonlinear Dynamical Systems Using an EM Algorithm , 1998, NIPS.

[36]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[37]  I. Mezić,et al.  Analysis of Fluid Flows via Spectral Properties of the Koopman Operator , 2013 .

[38]  Steven L. Brunton,et al.  Deep learning for universal linear embeddings of nonlinear dynamics , 2017, Nature Communications.

[39]  Kazuyuki Aihara,et al.  Reconstructing state spaces from multivariate data using variable delays. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[40]  Steven L. Brunton,et al.  Generalizing Koopman Theory to Allow for Inputs and Control , 2016, SIAM J. Appl. Dyn. Syst..

[41]  I. Mezić,et al.  Spectral analysis of nonlinear flows , 2009, Journal of Fluid Mechanics.

[42]  Yoshihiko Susuki,et al.  A prony approximation of Koopman Mode Decomposition , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[43]  P. Schmid,et al.  Dynamic mode decomposition of numerical and experimental data , 2008, Journal of Fluid Mechanics.

[44]  Bingni W. Brunton,et al.  Extracting spatial–temporal coherent patterns in large-scale neural recordings using dynamic mode decomposition , 2014, Journal of Neuroscience Methods.

[45]  Ioannis G Kevrekidis,et al.  Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator. , 2017, Chaos.

[46]  Joshua L. Proctor,et al.  Discovering dynamic patterns from infectious disease data using dynamic mode decomposition , 2015, International health.

[47]  Steven L. Brunton,et al.  Koopman Invariant Subspaces and Finite Linear Representations of Nonlinear Dynamical Systems for Control , 2015, PloS one.

[48]  Maximilian Karl,et al.  Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data , 2016, ICLR.

[49]  M. Mackey,et al.  Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics , 1998 .

[50]  O. Rössler.  An equation for continuous chaos , 1976 .

[51]  Clarence W. Rowley,et al.  Variants of Dynamic Mode Decomposition: Boundary Condition, Koopman, and Fourier Analyses , 2012, J. Nonlinear Sci..

[52]  Martin A. Riedmiller,et al.  Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images , 2015, NIPS.

[53]  Dimitris Kugiumtzis,et al.  Non-uniform state space reconstruction and coupling detection , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[54]  Steven L. Brunton,et al.  Dynamic Mode Decomposition with Control , 2014, SIAM J. Appl. Dyn. Syst..

[55]  I. Mezić,et al.  Applied Koopmanism. , 2012, Chaos.