Dynamic mode decomposition in vector-valued reproducing kernel Hilbert spaces for extracting dynamical structure among observables

Understanding nonlinear dynamical systems (NLDSs) is challenging in a variety of engineering and scientific fields. Dynamic mode decomposition (DMD), which is a numerical algorithm for the spectral analysis of Koopman operators, has been attracting attention as a way of obtaining global modal descriptions of NLDSs without requiring explicit prior knowledge. However, since existing DMD algorithms are in principle formulated based on the concatenation of scalar observables, it is not directly applicable to data with dependent structures among observables, which take, for example, the form of a sequence of graphs. In this paper, we formulate Koopman spectral analysis for NLDSs with structures among observables and propose an estimation algorithm for this problem. This method can extract and visualize the underlying low-dimensional global dynamics of NLDSs with structures among observables from data, which can be useful in understanding the underlying dynamics of such NLDSs. To this end, we first formulate the problem of estimating spectra of the Koopman operator defined in vector-valued reproducing kernel Hilbert spaces, and then develop an estimation procedure for this problem by reformulating tensor-based DMD. As a special case of our method, we propose the method named as Graph DMD, which is a numerical algorithm for Koopman spectral analysis of graph dynamical systems, using a sequence of adjacency matrices. We investigate the empirical performance of our method by using synthetic and real-world data.

[1]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[2]  Stefan Klus,et al.  Tensor-based dynamic mode decomposition , 2016, Nonlinearity.

[3]  U. Feige,et al.  Spectral Graph Theory , 2015 .

[4]  Vicsek,et al.  Novel type of phase transition in a system of self-driven particles. , 1995, Physical review letters.

[5]  Naoya Takeishi,et al.  Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition , 2017, NIPS.

[6]  Kilian Q. Weinberger,et al.  Graph Laplacian Regularization for Large-Scale Semidefinite Programming , 2006, NIPS.

[7]  K. Fujii,et al.  Resilient help to switch and overlap hierarchical subsystems in a small human group , 2016, Scientific Reports.

[8]  P. Schmid,et al.  Dynamic mode decomposition of numerical and experimental data , 2008, Journal of Fluid Mechanics.

[9]  Bingni W. Brunton,et al.  Extracting spatial–temporal coherent patterns in large-scale neural recordings using dynamic mode decomposition , 2014, Journal of Neuroscience Methods.

[10]  Daniel Kressner,et al.  A literature survey of low‐rank tensor approximation techniques , 2013, 1302.7121.

[11]  M. Macy,et al.  Complex Contagions and the Weakness of Long Ties1 , 2007, American Journal of Sociology.

[12]  Samarth Swarup,et al.  A general-purpose graph dynamical system modeling framework , 2011, Proceedings of the 2011 Winter Simulation Conference (WSC).

[13]  Naoya Takeishi,et al.  Data-driven spectral analysis for coordinative structures in periodic systems with unknown and redundant dynamics , 2019, bioRxiv.

[14]  Keisuke Fujii,et al.  Supervised dynamic mode decomposition via multitask learning , 2019, Pattern Recognit. Lett..

[15]  Yoshihiko Susuki,et al.  Nonlinear Koopman modes and power system stability assessment without models , 2014, 2014 IEEE PES General Meeting | Conference & Exposition.

[16]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[17]  I. Couzin,et al.  Collective memory and spatial sorting in animal groups. , 2002, Journal of theoretical biology.

[18]  I. Mezić,et al.  Spectral analysis of nonlinear flows , 2009, Journal of Fluid Mechanics.

[19]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[20]  Massimiliano Pontil,et al.  Regularized multi--task learning , 2004, KDD.

[21]  Hisashi Kashima,et al.  Eigenspace-based anomaly detection in computer systems , 2004, KDD.

[22]  Charles A. Micchelli,et al.  Universal Multi-Task Kernels , 2008, J. Mach. Learn. Res..

[23]  Naoya Takeishi,et al.  Sparse nonnegative dynamic mode decomposition , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[24]  Xavier Bresson,et al.  Structured Sequence Modeling with Graph Convolutional Recurrent Networks , 2016, ICONIP.

[25]  Charles A. Micchelli,et al.  On Learning Vector-Valued Functions , 2005, Neural Computation.

[26]  Charles A. Micchelli,et al.  Kernels for Multi--task Learning , 2004, NIPS.

[27]  Gabriel Taubin,et al.  A signal processing approach to fair surface design , 1995, SIGGRAPH.

[28]  Clarence W. Rowley,et al.  A Data–Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition , 2014, Journal of Nonlinear Science.

[29]  Keisuke Fujii,et al.  Koopman Spectral Kernels for Comparing Complex Dynamics: Application to Multiagent Sport Plays , 2017, ECML/PKDD.

[30]  I. Mezić Spectral Properties of Dynamical Systems, Model Reduction and Decompositions , 2005 .

[31]  Yoshinobu Kawahara,et al.  Dynamic Mode Decomposition with Reproducing Kernels for Koopman Spectral Analysis , 2016, NIPS.

[32]  Koh Takeuchi,et al.  Structurally Regularized Non-negative Tensor Factorization for Spatio-Temporal Pattern Discoveries , 2017, ECML/PKDD.

[33]  Charles A. Micchelli,et al.  Learning Multiple Tasks with Kernel Methods , 2005, J. Mach. Learn. Res..

[34]  Shilpa Chakravartula,et al.  Complex Networks: Structure and Dynamics , 2014 .

[35]  O. Sporns,et al.  Complex brain networks: graph theoretical analysis of structural and functional systems , 2009, Nature Reviews Neuroscience.

[36]  Zhizhen Zhao,et al.  Spatiotemporal Pattern Extraction by Spectral Analysis of Vector-Valued Observables , 2017, Journal of Nonlinear Science.

[37]  Mihaela van der Schaar,et al.  Bayesian Inference of Individualized Treatment Effects using Multi-task Gaussian Processes , 2017, NIPS.

[38]  Christian M. Reidys,et al.  An Introduction to Sequential Dynamical Systems , 2007, Universitext.

[39]  Sung Ha Kang,et al.  Image and Video Colorization Using Vector-Valued Reproducing Kernel Hilbert Spaces , 2010, Journal of Mathematical Imaging and Vision.

[40]  Mikhail Prokopenko,et al.  An Information Criterion for Inferring Coupling of Distributed Dynamical Systems , 2016, Front. Robot. AI.

[41]  Steven L. Brunton,et al.  Dynamic Mode Decomposition with Control , 2014, SIAM J. Appl. Dyn. Syst..

[42]  Steven L. Brunton,et al.  Multiresolution Dynamic Mode Decomposition , 2015, SIAM J. Appl. Dyn. Syst..

[43]  I. Mezic,et al.  Nonlinear Koopman Modes and Coherency Identification of Coupled Swing Dynamics , 2011, IEEE Transactions on Power Systems.

[44]  Ke Lu,et al.  $p$-Laplacian Regularized Sparse Coding for Human Activity Recognition , 2016, IEEE Transactions on Industrial Electronics.

[45]  Yasuo Tabei,et al.  Bayesian Dynamic Mode Decomposition , 2017, IJCAI.

[46]  Christian M. Reidys,et al.  Discrete, sequential dynamical systems , 2001, Discret. Math..

[47]  Neil D. Lawrence,et al.  Kernels for Vector-Valued Functions: a Review , 2011, Found. Trends Mach. Learn..

[48]  B. O. Koopman,et al.  Hamiltonian Systems and Transformation in Hilbert Space. , 1931, Proceedings of the National Academy of Sciences of the United States of America.

[49]  Steven L. Brunton,et al.  On dynamic mode decomposition: Theory and applications , 2013, 1312.0041.

[50]  Yoshinobu Kawahara,et al.  Prediction and classification in equation-free collective motion dynamics , 2018, PLoS Comput. Biol..

[51]  Yoshinobu Kawahara,et al.  Automatically recognizing strategic cooperative behaviors in various situations of a team sport , 2018, PloS one.

[52]  C. Wu Synchronization in networks of nonlinear dynamical systems coupled via a directed graph , 2005 .

[53]  Joshua L. Proctor,et al.  Discovering dynamic patterns from infectious disease data using dynamic mode decomposition , 2015, International health.

[54]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[55]  Naoya Takeishi,et al.  Subspace dynamic mode decomposition for stochastic Koopman analysis. , 2017, Physical review. E.