Discriminant Dynamic Mode Decomposition for Labeled Spatio-Temporal Data Collections

Extracting coherent patterns is one of the standard approaches towards understanding spatiotemporal data. Dynamic mode decomposition (DMD) is a powerful tool for extracting coherent patterns, but the original DMD and most of its variants do not consider label information, which is often available as side information of spatio-temporal data. In this work, we propose a new method for extracting distinctive coherent patterns from labeled spatio-temporal data collections, such that they contribute to major differences in a labeled set of dynamics. We achieve such pattern extraction by incorporating discriminant analysis into DMD. To this end, we define a kernel function on subspaces spanned by sets of dynamic modes and develop an objective to take both reconstruction goodness as DMD and class-separation goodness as discriminant analysis into account. We illustrate our method using a synthetic dataset and several real-world datasets. The proposed method can be a useful tool for exploratory data analysis for understanding spatio-temporal data.

[1]  Aleix M. Martínez,et al.  Kernel Optimization in Discriminant Analysis , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Bingni W. Brunton,et al.  Extracting spatial–temporal coherent patterns in large-scale neural recordings using dynamic mode decomposition , 2014, Journal of Neuroscience Methods.

[3]  J. Nathan Kutz,et al.  Variable Projection Methods for an Optimized Dynamic Mode Decomposition , 2017, SIAM J. Appl. Dyn. Syst..

[4]  Keisuke Fujii,et al.  Supervised dynamic mode decomposition via multitask learning , 2019, Pattern Recognit. Lett..

[5]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[6]  C. Caramanis What is ergodic theory , 1963 .

[7]  I. Mezić Spectral Properties of Dynamical Systems, Model Reduction and Decompositions , 2005 .

[8]  Todd D. Murphey,et al.  Model-Based Control Using Koopman Operators , 2017, Robotics: Science and Systems.

[9]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[10]  Zohreh Azimifar,et al.  Supervised principal component analysis: Visualization, classification and regression on subspaces and submanifolds , 2011, Pattern Recognit..

[11]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[12]  Clarence W. Rowley,et al.  A Data–Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition , 2014, Journal of Nonlinear Science.

[13]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[14]  Matthew T. Kaufman,et al.  Neural population dynamics during reaching , 2012, Nature.

[15]  Changsong Liu,et al.  Discriminative Transformation for Multi-Dimensional Temporal Sequences , 2017, IEEE Transactions on Image Processing.

[16]  I. Mezić,et al.  Applied Koopmanism. , 2012, Chaos.

[17]  Joshua L. Proctor,et al.  Discovering dynamic patterns from infectious disease data using dynamic mode decomposition , 2015, International health.

[18]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[19]  R. Fisher THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[20]  K. Tsuchiya,et al.  Variant and invariant patterns embedded in human locomotion through whole body kinematic coordination , 2010, Experimental Brain Research.

[21]  D. Giannakis Data-driven spectral decomposition and forecasting of ergodic dynamical systems , 2015, Applied and Computational Harmonic Analysis.

[22]  Igor Mezic,et al.  On Convergence of Extended Dynamic Mode Decomposition to the Koopman Operator , 2017, J. Nonlinear Sci..

[23]  S. Wold,et al.  PLS-regression: a basic tool of chemometrics , 2001 .

[24]  Bryan Eisenhower,et al.  CREATING ZONING APPROXIMATIONS TO BUILDING ENERGY MODELS USING THE KOOPMAN OPERATOR , 2012 .

[25]  Peter J. Schmid,et al.  Sparsity-promoting dynamic mode decomposition , 2012, 1309.4165.

[26]  Gene H. Golub,et al.  The differentiation of pseudo-inverses and non-linear least squares problems whose variables separate , 1972, Milestones in Matrix Computation.

[27]  P. Schmid,et al.  Dynamic mode decomposition of numerical and experimental data , 2008, Journal of Fluid Mechanics.

[28]  Yoshihiko Susuki,et al.  Nonlinear Koopman modes and power system stability assessment without models , 2014, 2014 IEEE PES General Meeting | Conference & Exposition.

[29]  Steven L. Brunton,et al.  On dynamic mode decomposition: Theory and applications , 2013, 1312.0041.

[30]  Michael I. Jordan,et al.  Kernel dimension reduction in regression , 2009, 0908.1854.

[31]  Xiaoqing Ding,et al.  Discriminative Dimensionality Reduction for Multi-Dimensional Sequences , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Keisuke Fujii,et al.  Data-driven spectral analysis for coordinative structures in periodic human locomotion , 2019, Scientific Reports.

[33]  J R Wolpaw,et al.  Spatial filter selection for EEG-based communication. , 1997, Electroencephalography and clinical neurophysiology.

[34]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[35]  Dimitrios Giannakis,et al.  Delay-Coordinate Maps and the Spectra of Koopman Operators , 2017, Journal of Statistical Physics.

[36]  I. Mezić,et al.  Spectral analysis of nonlinear flows , 2009, Journal of Fluid Mechanics.

[37]  Daniel D. Lee,et al.  Grassmann discriminant analysis: a unifying view on subspace-based learning , 2008, ICML '08.

[38]  Igor Mezic,et al.  Ergodic Theory, Dynamic Mode Decomposition, and Computation of Spectral Properties of the Koopman Operator , 2016, SIAM J. Appl. Dyn. Syst..

[39]  Clarence W. Rowley,et al.  Variants of Dynamic Mode Decomposition: Boundary Condition, Koopman, and Fourier Analyses , 2012, J. Nonlinear Sci..

[40]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[41]  Ying Wu,et al.  Learning Low-Dimensional Temporal Representations , 2018, ICML.

[42]  Luis M. Candanedo,et al.  Data driven prediction models of energy use of appliances in a low-energy house , 2017 .

[43]  Steven L. Brunton,et al.  Dynamic mode decomposition - data-driven modeling of complex systems , 2016 .