Tipping Point Detection Using Reservoir Computing

Detection in high fidelity of tipping points, the emergence of which is often induced by invisible changes in internal structures or/and external interferences, is paramountly beneficial to understanding and predicting complex dynamical systems (CDSs). Detection approaches, which have been fruitfully developed from several perspectives (e.g., statistics, dynamics, and machine learning), have their own advantages but still encounter difficulties in the face of high-dimensional, fluctuating datasets. Here, using the reservoir computing (RC), a recently notable, resource-conserving machine learning method for reconstructing and predicting CDSs, we articulate a model-free framework to accomplish the detection only using the time series observationally recorded from the underlying unknown CDSs. Specifically, we encode the information of the CDS in consecutive time durations of finite length into the weights of the readout layer in an RC, and then we use the learned weights as the dynamical features and establish a mapping from these features to the system’s changes. Our designed framework can not only efficiently detect the changing positions of the system but also accurately predict the intensity change as the intensity information is available in the training data. We demonstrate the efficacy of our supervised framework using the dataset produced by representative physical, biological, and real-world systems, showing that our framework outperforms those traditional methods on the short-term data produced by the time-varying or/and noise-perturbed systems. We believe that our framework, on one hand, complements the major functions of the notable RC intelligent machine and, on the other hand, becomes one of the indispensable methods for deciphering complex systems.

[1]  Huanfei Ma,et al.  Embedding Theory of Reservoir Computing and Reducing Reservoir Network Using Time Delays , 2023, Physical Review Research.

[2]  Huanfei Ma,et al.  Continuity Scaling: A Rigorous Framework for Detecting and Quantifying Causality Accurately , 2022, Research.

[3]  Jie Sun,et al.  Harvesting random embedding for high-frequency change-point detection in temporal complex systems , 2021, National science review.

[4]  Gouhei Tanaka,et al.  Transfer-RLS method and transfer-FORCE learning for simple and fast training of reservoir computing models , 2021, Neural Networks.

[5]  T. Carroll Optimizing Reservoir Computers for Signal Classification , 2021, Frontiers in Physiology.

[6]  Adrian S. Wong,et al.  Robust forecasting using predictive generalized synchronization in reservoir computing. , 2021, Chaos.

[7]  Alexander Bertrand,et al.  Change Point Detection in Time Series Data Using Autoencoders With a Time-Invariant Representation , 2020, IEEE Transactions on Signal Processing.

[8]  D. Gu'ery-Odelin,et al.  Accuracy of neural networks for the simulation of chaotic dynamics: precision of training data vs precision of the algorithm , 2020, Chaos.

[9]  J. Kurths,et al.  Partial cross mapping eliminates indirect causal influences , 2020, Nature Communications.

[10]  T. Lenton Tipping positive change , 2020, Philosophical Transactions of the Royal Society B.

[11]  Amitava Banerjee,et al.  Using Machine Learning to Assess Short Term Causal Dependence and Infer Network Links , 2019, Chaos.

[12]  Jaideep Pathak,et al.  Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics , 2019, Neural Networks.

[13]  Seung Hwan Lee,et al.  Temporal data classification and forecasting using a memristor-based reservoir computing system , 2019, Nature Electronics.

[14]  Aditya Gilra,et al.  Dynamical Learning of Dynamics. , 2019, Physical review letters.

[15]  Toshiyuki Yamane,et al.  Recent Advances in Physical Reservoir Computing: A Review , 2018, Neural Networks.

[16]  Robert Jenssen,et al.  Reservoir Computing Approaches for Representation and Classification of Multivariate Time Series , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Jaideep Pathak,et al.  Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach. , 2018, Physical review letters.

[18]  Masahide Kaneko,et al.  Segmenting Continuous Motions with Hidden Semi-markov Models and Gaussian Processes , 2017, Front. Neurorobot..

[19]  Niall M. Adams,et al.  Continuous monitoring for changepoints in data streams using adaptive estimation , 2017, Stat. Comput..

[20]  Diane J. Cook,et al.  A survey of methods for time series change point detection , 2017, Knowledge and Information Systems.

[21]  P. Fearnhead,et al.  Computationally Efficient Changepoint Detection for a Range of Penalties , 2017 .

[22]  Luonan Chen,et al.  Part mutual information for quantifying direct associations in networks , 2016, Proceedings of the National Academy of Sciences.

[23]  Bien Aik Tan,et al.  Change-point detection for recursive Bayesian geoacoustic inversions. , 2015, The Journal of the Acoustical Society of America.

[24]  Axel Munk,et al.  Multiscale DNA partitioning: statistical evidence for segments , 2014, Bioinform..

[25]  Robert Lund,et al.  Changepoint Detection in Climate Time Series with Long-Term Trends , 2013 .

[26]  A. Munk,et al.  Multiscale change point inference , 2013, 1301.7212.

[27]  Young-Koo Lee,et al.  Comprehensive Context Recognizer Based on Multimodal Sensors in a Smartphone , 2012, Sensors.

[28]  Masashi Sugiyama,et al.  Sequential change‐point detection based on direct density‐ratio estimation , 2012, Stat. Anal. Data Min..

[29]  Nigel Collier,et al.  Change-Point Detection in Time-Series Data by Relative Density-Ratio Estimation , 2012, Neural Networks.

[30]  Sylvain Arlot,et al.  A Kernel Multiple Change-point Algorithm via Model Selection , 2012, J. Mach. Learn. Res..

[31]  Deborah Estrin,et al.  Using mobile phones to determine transportation modes , 2010, TOSN.

[32]  Olivier Cappé,et al.  Distributed detection/localization of change-points in high-dimensional network traffic data , 2009, Statistics and Computing.

[33]  Yoshinobu Kawahara,et al.  Change-Point Detection in Time-Series Data Based on Subspace Identification , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[34]  O. Cappé,et al.  Retrospective Mutiple Change-Point Estimation with Kernels , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[35]  Li Wei,et al.  Semi-supervised time series classification , 2006, KDD '06.

[36]  Nils Bertschinger,et al.  Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks , 2004, Neural Computation.

[37]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[38]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[39]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[40]  Oreoluwa Alao,et al.  Discovering Dynamical Parameters by Interpreting Echo State Networks , 2021 .

[41]  H. Jaeger A tutorial on training recurrent neural networks, covering BPTT, RTRL, EKF and the "echo state network" approach , 2021 .

[42]  N. Vayatis,et al.  Selective review of offline change point detection methods , 2019 .

[43]  Xing Xie,et al.  Understanding transportation modes based on GPS data for web applications , 2010, TWEB.

[44]  Ali H. Shoeb,et al.  Application of machine learning to epileptic seizure onset detection and treatment , 2009 .

[45]  Benjamin Schrauwen,et al.  Reservoir Computing with Stochastic Bitstream Neurons , 2005 .

[46]  L. Breiman Random Forests , 2001, Machine Learning.

[47]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .