Learning dynamical systems from data: a simple cross-validation perspective

Regressing the vector field of a dynamical system from a finite number of observed states is a natural way to learn surrogate models for such systems. We present variants of cross-validation (Kernel Flows [31] and its variants based on Maximum Mean Discrepancy and Lyapunov exponents) as simple approaches for learning the kernel used in these emulators.

[1]  G. Santin,et al.  Kernel Methods for Surrogate Modeling , 2019, System- and Data-Driven Methods and Algorithms.

[2]  Jake Bouvrie,et al.  Balanced reduction of nonlinear control systems in reproducing kernel Hilbert space , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[3]  Jaideep Pathak,et al.  Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data. , 2017, Chaos.

[4]  Andrew M. Stuart,et al.  Consistency of Empirical Bayes And Kernel Flow For Hierarchical Parameter Estimation , 2020, Math. Comput..

[5]  Devika Subramanian,et al.  Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM , 2019 .

[6]  J. Guckenheimer.  ONE‐DIMENSIONAL DYNAMICS * , 1980 .

[7]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[8]  Zongmin Wu,et al.  Local error estimates for radial basis function interpolation of scattered data , 1993 .

[9]  Houman Owhadi,et al.  Deep regularization and direct training of the inner layers of Neural Networks with Kernel Flows , 2020, ArXiv.

[10]  Edward Ott,et al.  Attractor reconstruction by machine learning. , 2018, Chaos.

[11]  Stefan Klus,et al.  Data-driven approximation of the Koopman generator: Model reduction, system identification, and control , 2019 .

[12]  Houman Owhadi,et al.  Kernel Mode Decomposition and programmable/interpretable regression networks , 2019, ArXiv.

[13]  J. A. Stewart,et al.  Nonlinear Time Series Analysis , 2015 .

[14]  Houman Owhadi,et al.  Operator-Adapted Wavelets, Fast Solvers, and Numerical Homogenization , 2019 .

[15]  Eckmann,et al.  Liapunov exponents from time series. , 1986, Physical review. A, General physics.

[16]  Stefan Klus,et al.  Dimensionality Reduction of Complex Metastable Systems via Kernel Embeddings of Transition Manifolds , 2021, J. Nonlinear Sci..

[17]  Robert Schaback,et al.  Bases for kernel-based spaces , 2011, J. Comput. Appl. Math..

[18]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[19]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[20]  George E. P. Box,et al.  Time Series Analysis: Forecasting and Control , 1977 .

[21]  Gunnar Rätsch,et al.  Predicting Time Series with Support Vector Machines , 1997, ICANN.

[22]  F. Girosi,et al.  Nonlinear prediction of chaotic time series using support vector machines , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[23]  Houman Owhadi,et al.  Bayesian Numerical Homogenization , 2014, Multiscale Model. Simul..

[24]  Feliks Nüske,et al.  Kernel-Based Approximation of the Koopman Generator and Schrödinger Operator , 2020, Entropy.

[25]  Martin Casdagli,et al.  Nonlinear prediction of chaotic time series , 1989 .

[26]  Liva Ralaivola,et al.  Dynamical Modeling with Kernels for Nonlinear Time Series Prediction , 2003, NIPS.

[27]  Mehmet Emre Çek,et al.  Analysis of observed chaotic data , 2004 .

[28]  Steven L. Brunton,et al.  Chaos as an intermittently forced linear system , 2016, Nature Communications.

[29]  P. Giesl,et al.  Approximation of Lyapunov functions from noisy data , 2016, Journal of Computational Dynamics.

[30]  Robert H. Shumway,et al.  Time series analysis and its applications : with R examples , 2017 .

[31]  D. Giannakis,et al.  Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques , 2019, Physica D: Nonlinear Phenomena.

[32]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[33]  Houman Owhadi,et al.  Kernel Flows: from learning kernels from data into the abyss , 2018, J. Comput. Phys..

[34]  Houman Owhadi,et al.  Sparse Cholesky factorization by Kullback-Leibler minimization , 2020, SIAM J. Sci. Comput..

[35]  Bernard Haasdonk,et al.  Greedy Kernel Methods for Center Manifold Approximation , 2018, 1810.11329.

[36]  JAKE BOUVRIE,et al.  Kernel Methods for the Approximation of Nonlinear Systems , 2011, SIAM J. Control. Optim..

[37]  Devika Subramanian,et al.  Data-driven prediction of a multi-scale Lorenz 96 chaotic system using a hierarchy of deep learning methods: Reservoir computing, ANN, and RNN-LSTM , 2019, ArXiv.

[38]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .