Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator.

Numerical approximation methods for the Koopman operator have advanced considerably in the last few years. In particular, data-driven approaches such as dynamic mode decomposition (DMD)51 and its generalization, the extended-DMD (EDMD), are becoming increasingly popular in practical applications. The EDMD improves upon the classical DMD by the inclusion of a flexible choice of dictionary of observables which spans a finite dimensional subspace on which the Koopman operator can be approximated. This enhances the accuracy of the solution reconstruction and broadens the applicability of the Koopman formalism. Although the convergence of the EDMD has been established, applying the method in practice requires a careful choice of the observables to improve convergence with just a finite number of terms. This is especially difficult for high dimensional and highly nonlinear systems. In this paper, we employ ideas from machine learning to improve upon the EDMD method. We develop an iterative approximation algorithm which couples the EDMD with a trainable dictionary represented by an artificial neural network. Using the Duffing oscillator and the Kuramoto Sivashinsky partical differential equation as examples, we show that our algorithm can effectively and efficiently adapt the trainable dictionary to the problem at hand to achieve good reconstruction accuracy without the need to choose a fixed dictionary a priori. Furthermore, to obtain a given accuracy, we require fewer dictionary terms than EDMD with fixed dictionaries. This alleviates an important shortcoming of the EDMD algorithm and enhances the applicability of the Koopman framework to practical problems.

[1]  Frank Noé,et al.  Variational Koopman models: Slow collective variables and molecular kinetics from short off-equilibrium simulations. , 2016, The Journal of chemical physics.

[2]  B. O. Koopman,et al.  Hamiltonian Systems and Transformation in Hilbert Space. , 1931, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Steven L. Brunton,et al.  On dynamic mode decomposition: Theory and applications , 2013, 1312.0041.

[4]  I. Kevrekidis,et al.  Approximate inertial manifolds for the Kuramoto-Sivashinsky equation: analysis and computations , 1990 .

[5]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[6]  I. Mezić,et al.  Isostables, isochrons, and Koopman spectrum for the action-angle representation of stable fixed point dynamics , 2013, 1302.0032.

[7]  I. Mezić,et al.  Applied Koopmanism. , 2012, Chaos.

[8]  Steven L. Brunton,et al.  Koopman Invariant Subspaces and Finite Linear Representations of Nonlinear Dynamical Systems for Control , 2015, PloS one.

[9]  Michael Elad,et al.  Sparse and Redundant Modeling of Image Content Using an Image-Signature-Dictionary , 2008, SIAM J. Imaging Sci..

[10]  J. Neumann Zur Operatorenmethode In Der Klassischen Mechanik , 1932 .

[11]  Kari Karhunen,et al.  Über lineare Methoden in der Wahrscheinlichkeitsrechnung , 1947 .

[12]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[13]  Matthew O. Williams,et al.  A Kernel-Based Approach to Data-Driven Koopman Spectral Analysis , 2014, 1411.2260.

[14]  Clarence W. Rowley,et al.  A Data–Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition , 2014, Journal of Nonlinear Science.

[15]  Igor Mezic,et al.  Building energy modeling: A systematic approach to zoning and model reduction using Koopman Mode Analysis , 2015 .

[16]  I. Mezić Spectral Properties of Dynamical Systems, Model Reduction and Decompositions , 2005 .

[17]  I. Mezić,et al.  Analysis of Fluid Flows via Spectral Properties of the Koopman Operator , 2013 .

[18]  R. Téman,et al.  Integral Manifolds and Inertial Manifolds for Dissipative Partial Differential Equations , 1988 .

[19]  Kjersti Engan,et al.  Method of optimal directions for frame design , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[20]  Igor Mezic,et al.  Koopman Operator Spectrum and Data Analysis , 2017, 1702.07597.

[21]  A. Kolmogoroff Über die analytischen Methoden in der Wahrscheinlichkeitsrechnung , 1931 .

[22]  Clarence W. Rowley,et al.  Data fusion via intrinsic dynamic variables: An application of data-driven Koopman spectral analysis , 2014, 1411.5424.

[23]  Zhizhen Zhao,et al.  Spatiotemporal Feature Extraction with Data-Driven Koopman Operators , 2015, FE@NIPS.

[24]  Igor Mezi'c,et al.  Correspondence between Koopman mode decomposition, resolvent mode decomposition, and invariant solutions of the Navier-Stokes equations , 2016, 1603.08378.

[25]  Hao Wu,et al.  Data-Driven Model Reduction and Transfer Operator Approximation , 2017, J. Nonlinear Sci..

[26]  Huibert Kwakernaak,et al.  Linear Optimal Control Systems , 1972 .

[27]  D. Giannakis Data-driven spectral decomposition and forecasting of ergodic dynamical systems , 2015, Applied and Computational Harmonic Analysis.

[28]  Yann LeCun,et al.  Convolutional networks and applications in vision , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[29]  I. Mezić,et al.  Spectral analysis of nonlinear flows , 2009, Journal of Fluid Mechanics.

[30]  P. Schmid,et al.  Dynamic mode decomposition of numerical and experimental data , 2008, Journal of Fluid Mechanics.

[31]  Bingni W. Brunton,et al.  Extracting spatial–temporal coherent patterns in large-scale neural recordings using dynamic mode decomposition , 2014, Journal of Neuroscience Methods.

[32]  A. Ng Feature selection, L1 vs. L2 regularization, and rotational invariance , 2004, Twenty-first international conference on Machine learning - ICML '04.

[33]  Andrzej Banaszuk,et al.  Comparison of systems with complex behavior , 2004 .

[34]  J. Neumann,et al.  Operator Methods in Classical Mechanics, II , 1942 .

[35]  D. L. Donoho,et al.  Compressed sensing , 2006, IEEE Trans. Inf. Theory.