Metric on random dynamical systems with vector-valued reproducing kernel Hilbert spaces

Development of metrics for structural data-generating mechanisms is fundamental in machine learning and the related fields. In this paper, we give a general framework to construct metrics on random nonlinear dynamical systems, defined with the Perron-Frobenius operators in vector-valued reproducing kernel Hilbert spaces (vvRKHSs). We employ vvRKHSs to design mathematically manageable metrics and also to introduce operator-valued kernels, which enables us to handle randomness in systems. Our metric provides an extension of the existing metrics for deterministic systems, and gives a specification of the kernel maximal mean discrepancy of random processes. Moreover, by considering the time-wise independence of random processes, we clarify a connection between our metric and the independence criteria with kernels such as Hilbert-Schmidt independence criteria. We empirically illustrate our metric with synthetic data, and evaluate it in the context of the independence test for random processes. We also evaluate the performance with real time seris datas via clusering tasks.

[1]  Bernhard Schölkopf,et al.  Measuring Statistical Dependence with Hilbert-Schmidt Norms , 2005, ALT.

[2]  Arthur Gretton,et al.  A Kernel Independence Test for Random Processes , 2014, ICML.

[3]  Igor Mezic On Comparison of Dynamics of Dissipative and Finite-Time Systems Using Koopman Operator Methods* , 2016 .

[4]  Keisuke Fujii,et al.  Koopman Spectral Kernels for Comparing Complex Dynamics: Application to Multiagent Sport Plays , 2017, ECML/PKDD.

[5]  P. Billingsley,et al.  Convergence of Probability Measures , 1970, The Mathematical Gazette.

[6]  V. Araújo Random Dynamical Systems , 2006, math/0608162.

[7]  Keisuke Fujii,et al.  Metric on Nonlinear Dynamical Systems with Perron-Frobenius Operators , 2018, NeurIPS.

[8]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[9]  Igor Mezic,et al.  Koopman Operator Spectrum for Random Dynamical Systems , 2017, Journal of Nonlinear Science.

[10]  Bart De Moor,et al.  Subspace angles between ARMA models , 2002, Syst. Control. Lett..

[11]  Kenji Fukumizu,et al.  Universality, Characteristic Kernels and RKHS Embedding of Measures , 2010, J. Mach. Learn. Res..

[12]  Alexander J. Smola,et al.  Binet-Cauchy Kernels on Dynamical Systems and its Application to the Analysis of Dynamic Scenes , 2007, International Journal of Computer Vision.

[13]  Alison L Gibbs,et al.  On Choosing and Bounding Probability Metrics , 2002, math/0209021.

[14]  Andrzej Banaszuk,et al.  Comparison of systems with complex behavior , 2004 .

[15]  Keisuke Fujii,et al.  Dynamic mode decomposition in vector-valued reproducing kernel Hilbert spaces for extracting dynamical structure among observables , 2018, Neural Networks.

[16]  C. Carmeli,et al.  VECTOR VALUED REPRODUCING KERNEL HILBERT SPACES OF INTEGRABLE FUNCTIONS AND MERCER THEOREM , 2006 .

[17]  Richard J. Martin A metric for ARMA processes , 2000, IEEE Trans. Signal Process..