Inferring untrained complex dynamics of delay systems using an adapted echo state network

Caused by finite signal propagation velocities, many complex systems feature time delays that may induce high-dimensional chaotic behavior and make forecasting intricate. Here, we propose an echo state network adaptable to the physics of systems with arbitrary delays. After training the network to forecast a system with a unique and sufficiently long delay, it already learned to predict the system dynamics for all other delays. A simple adaptation of the network’s topology allows us to infer untrained features such as high-dimensional chaotic attractors, bifurcations, and even multistabilities, that emerge with shorter and longer delays. Thus, the fusion of physical knowledge of the delay system and data-driven machine learning yields a model with high generalization capabilities and unprecedented prediction accuracy.

[1]  Huanfei Ma,et al.  Detecting unstable periodic orbits based only on time series: When adaptive delayed feedback control meets reservoir computing. , 2019, Chaos.

[2]  Felix Köster,et al.  Limitations of the recall capabilities in delay based reservoir computing systems , 2020, Cognitive Computation.

[3]  W. Hahn,et al.  Biologically informed deep neural network for prostate cancer discovery , 2021, Nature.

[4]  Y. Pesin CHARACTERISTIC LYAPUNOV EXPONENTS AND SMOOTH ERGODIC THEORY , 1977 .

[5]  Laurent Larger,et al.  Virtual chimera states for delayed-feedback systems. , 2013, Physical review letters.

[6]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[7]  Serge Massar,et al.  Brain-Inspired Photonic Signal Processor for Generating Periodic Patterns and Emulating Chaotic Systems , 2017 .

[8]  R. Toral,et al.  Analysis and characterization of the hyperchaos generated by a semiconductor laser subject to a delayed feedback loop , 2005, IEEE Journal of Quantum Electronics.

[9]  Serge Massar,et al.  Using a reservoir computer to learn chaotic attractors, with applications to chaos synchronisation and cryptography , 2018, Physical review. E.

[10]  Jaideep Pathak,et al.  Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach. , 2018, Physical review letters.

[11]  Benjamin Schrauwen,et al.  Reservoir Computing Trends , 2012, KI - Künstliche Intelligenz.

[12]  L. Appeltant,et al.  Information processing using a single dynamical node as complex system , 2011, Nature communications.

[13]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[14]  Holger Kantz,et al.  Identifying and Modeling Delay Feedback Systems. , 1998, chao-dyn/9907019.

[15]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[16]  Jaideep Pathak,et al.  Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data. , 2017, Chaos.

[17]  Ying-Cheng Lai,et al.  Detecting unstable periodic orbits in high-dimensional chaotic systems from time series: reconstruction meeting with adaptation. , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[18]  Gibbs,et al.  High-dimension chaotic attractors of a nonlinear ring cavity. , 1986, Physical review letters.

[19]  Edward Ott,et al.  Attractor reconstruction by machine learning. , 2018, Chaos.

[20]  T. Erneux Applied Delay Differential Equations , 2009 .

[21]  George J. Pappas,et al.  Teaching recurrent neural networks to infer global temporal structure from local examples , 2021, Nature Machine Intelligence.

[22]  Michelle Girvan,et al.  Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model , 2018, Chaos.

[23]  Y. Lai,et al.  Data Based Identification and Prediction of Nonlinear and Complex Dynamical Systems , 2016, 1704.08764.

[24]  Bhavin J. Shastri,et al.  Takens-inspired neuromorphic processor: a downsizing tool for random recurrent neural networks via feature extraction , 2019, Physical Review Research.

[25]  Laurent Larger,et al.  Ikeda Hopf bifurcation revisited , 2004 .

[26]  W. Kinzel,et al.  Strong and weak chaos in nonlinear networks with time-delayed couplings. , 2011, Physical review letters.

[27]  A. Wills,et al.  Physics-informed machine learning , 2021, Nature Reviews Physics.

[28]  Christoph Räth,et al.  Good and bad predictions: Assessing and improving the replication of chaotic attractors by means of reservoir computing. , 2019, Chaos.

[29]  Laurent Larger,et al.  Coupled nonlinear delay systems as deep convolutional neural networks , 2019, Physical review letters.

[30]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[31]  Celso Grebogi,et al.  Machine learning prediction of critical transition and system collapse , 2020, ArXiv.

[32]  Aditya Gilra,et al.  Dynamical Learning of Dynamics. , 2019, Physical review letters.

[33]  Fatihcan M Atay,et al.  Clustered chimera states in delay-coupled oscillator systems. , 2008, Physical review letters.

[34]  Thomas de Quincey [C] , 2000, The Works of Thomas De Quincey, Vol. 1: Writings, 1799–1820.

[35]  Daniel J. Gauthier,et al.  Model-free inference of unseen attractors: Reconstructing phase space features from a single noisy trajectory using reservoir computing , 2021, Chaos.

[36]  Tongfeng Weng,et al.  Mapping topological characteristics of dynamical systems into neural networks: A reservoir computing approach. , 2020, Physical review. E.

[37]  P. Alam ‘G’ , 2021, Composites Engineering: An A–Z Guide.

[38]  D. Lazer,et al.  The Parable of Google Flu: Traps in Big Data Analysis , 2014, Science.

[39]  Emergence of transient chaos and intermittency in machine learning , 2021, Journal of Physics: Complexity.

[40]  Ljupco Kocarev,et al.  Introduction to Focus Issue: When machine learning meets complex systems: Networks, chaos, and nonlinear dynamics. , 2020, Chaos.