Accuracy of neural networks for the simulation of chaotic dynamics: precision of training data vs precision of the algorithm

We explore the influence of precision of the data and the algorithm for the simulation of chaotic dynamics by neural network techniques. For this purpose, we simulate the Lorenz system with different precisions using three different neural network techniques adapted to time series, namely, reservoir computing [using Echo State Network (ESN)], long short-term memory, and temporal convolutional network, for both short- and long-time predictions, and assess their efficiency and accuracy. Our results show that the ESN network is better at predicting accurately the dynamics of the system, and that in all cases, the precision of the algorithm is more important than the precision of the training data for the accuracy of the predictions. This result gives support to the idea that neural networks can perform time-series predictions in many practical applications for which data are necessarily of limited precision, in line with recent results. It also suggests that for a given set of data, the reliability of the predictions can be significantly improved by using a network with higher precision than the one of the data.

[1]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[2]  Jaideep Pathak,et al.  Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics , 2019, Neural Networks.

[3]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[4]  Devika Subramanian,et al.  Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network , 2020, Nonlinear Processes in Geophysics.

[5]  Devika Subramanian,et al.  Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM , 2019 .

[6]  O. Rössler.  An equation for continuous chaos , 1976 .

[7]  Devika Subramanian,et al.  Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM , 2019 .

[8]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Dima L. Shepelyansky,et al.  Some statistical properties of simple classically stochastic quantum systems , 1983 .

[10]  Jaideep Pathak,et al.  Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data. , 2017, Chaos.

[11]  Edward Ott,et al.  Attractor reconstruction by machine learning. , 2018, Chaos.

[12]  Jaideep Pathak,et al.  Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach. , 2018, Physical review letters.

[13]  Vladlen Koltun,et al.  An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling , 2018, ArXiv.

[14]  Simon Portegies Zwart,et al.  Newton vs the machine: solving the chaotic three-body problem using deep neural networks , 2019, ArXiv.

[15]  E. Lorenz.  Deterministic nonperiodic flow , 1963 .

[16]  Jürgen Schmidhuber,et al.  Applying LSTM to Time Series Predictable through Time-Window Approaches , 2000, ICANN.

[17]  Petros Koumoutsakos,et al.  Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks , 2018, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[18]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[19]  Ljupco Kocarev,et al.  Introduction to Focus Issue: When machine learning meets complex systems: Networks, chaos, and nonlinear dynamics. , 2020, Chaos.

[20]  George Kurian,et al.  Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[21]  Shepelyansky,et al.  Dynamical stability of quantum "chaotic" motion in a hydrogen atom. , 1986, Physical review letters.

[22]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.