A Kernel Two-sample Test for Dynamical Systems

A kernel two-sample test is developed for deciding whether two dynamical systems are identical based on data streams from these systems. Such comparison of dynamical systems is relevant, for example, when evaluating model-based design, detecting anomalies in medical data, or for transferring knowledge from one system to another. Kernel two-sample tests are a well established statistical method for comparing probability distributions and have been applied to many diverse objects, yet rarely to dynamical systems. In this paper, we propose an extension of the kernel two-sample test to dynamical systems based on ergodicity theory. This measure-theoretical point of view on dynamical systems allows us to compare them in a meaningful way. In particular, we do not require synchronous sampling, identical initial conditions, or similar restrictive assumptions. We demonstrate the effectiveness of the proposed method experimentally on human walking data by detecting anomalies in walking patterns.

[1]  Heikki Haario,et al.  Componentwise adaptation for high dimensional MCMC , 2005, Comput. Stat..

[2]  Thomas B. Schön,et al.  System identification of nonlinear state-space models , 2011, Autom..

[3]  Nam Hee Kim,et al.  Learning to Correspond Dynamical Systems , 2020, L4DC.

[4]  M. Einsiedler,et al.  Ergodic Theory: with a view towards Number Theory , 2010 .

[5]  G. Birkhoff.  Proof of the Ergodic Theorem , 1931, Proceedings of the National Academy of Sciences.

[6]  Bernhard Schölkopf,et al.  Kernel Mean Embedding of Distributions: A Review and Beyonds , 2016, Found. Trends Mach. Learn..

[7]  Joonho Lee,et al.  Learning agile and dynamic motor skills for legged robots , 2019, Science Robotics.

[8]  Bernhard Schölkopf,et al.  A Permutation-Based Kernel Conditional Independence Test , 2014, UAI.

[9]  Igor Mezic.  On Comparison of Dynamics of Dissipative and Finite-Time Systems Using Koopman Operator Methods* , 2016 .

[10]  Zoubin Ghahramani,et al.  Statistical Model Criticism using Kernel Two Sample Tests , 2015, NIPS.

[11]  Arthur Gretton,et al.  A Kernel Independence Test for Random Processes , 2014, ICML.

[12]  Sivaraman Balakrishnan,et al.  Optimal kernel choice for large-scale two-sample tests , 2012, NIPS.

[13]  P. Walters.  Introduction to Ergodic Theory , 1977 .

[14]  Michael I. Jordan,et al.  Learning Without Mixing: Towards A Sharp Analysis of Linear System Identification , 2018, COLT.

[15]  Johan A. K. Suykens,et al.  Kernel Density Estimation for Dynamical Systems , 2016, J. Mach. Learn. Res..

[16]  Lennart Ljung,et al.  Nonlinear System Identification: A User-Oriented Road Map , 2019, IEEE Control Systems.

[17]  Bernhard Schölkopf,et al.  A New Distribution-Free Concept for Representing, Comparing, and Propagating Uncertainty in Dynamical Systems with Kernel Probabilistic Programming , 2019, ArXiv.

[18]  J. Doyle,et al.  Essentials of Robust Control , 1997 .

[19]  Wojciech Zaremba,et al.  B-test: A Non-parametric, Low Variance Kernel Two-sample Test , 2013, NIPS.

[20]  Alexander J. Smola,et al.  Binet-Cauchy Kernels on Dynamical Systems and its Application to the Analysis of Dynamic Scenes , 2007, International Journal of Computer Vision.

[21]  C. Caramanis.  What is ergodic theory , 1963 .

[22]  Arthur Gretton,et al.  A Wild Bootstrap for Degenerate Kernel Tests , 2014, NIPS.

[23]  S. Joe Qin,et al.  A survey of industrial model predictive control technology , 2003 .

[24]  Christian Brecher,et al.  Industrial Internet of Things and Cyber Manufacturing Systems , 2017 .

[25]  Angela P. Schoellig,et al.  Experience Selection Using Dynamics Similarity for Efficient Multi-Source Transfer Learning Between Robots , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[27]  Ian Melbourne,et al.  The Lorenz Attractor is Mixing , 2005 .

[28]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[29]  Keisuke Fujii,et al.  Metric on Nonlinear Dynamical Systems with Perron-Frobenius Operators , 2018, NeurIPS.

[30]  M. Mirzakhani,et al.  Introduction to Ergodic theory , 2010 .