Learning Chaotic Attractors by Neural Networks

An algorithm is introduced that trains a neural network to identify chaotic dynamics from a single measured time series. During training, the algorithm learns to short-term predict the time series. At the same time a criterion, developed by Diks, van Zwet, Takens, and de Goede (1996) is monitored that tests the hypothesis that the reconstructed attractors of model-generated and measured data are the same. Training is stopped when the prediction error is low and the model passes this test. Two other features of the algorithm are (1) the way the state of the system, consisting of delays from the time series, has its dimension reduced by weighted principal component analysis data reduction, and (2) the user-adjustable prediction horizon obtained by error propagationpartially propagating prediction errors to the next time step. The algorithm is first applied to data from an experimental-driven chaotic pendulum, of which two of the three state variables are known. This is a comprehensive example that shows how well the Diks test can distinguish between slightly different attractors. Second, the algorithm is applied to the same problem, but now one of the two known state variables is ignored. Finally, we present a model for the laser data from the Santa Fe time-series competition (set A). It is the first model for these data that is not only useful for short-term predictions but also generates time series with similar chaotic characteristics as the measured data.

[1]  Simon Haykin,et al.  Detection of signals in chaos , 1995, Proc. IEEE.

[2]  G. P. King,et al.  Extracting qualitative dynamics from experimental data , 1986 .

[3]  Abhay B. Bulsari,et al.  A recurrent network for modeling noisy temporal sequences , 1995, Neurocomputing.

[4]  C. Diks,et al.  Detecting differences between delay vector distributions. , 1996, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[5]  C. Lee Giles,et al.  Robust Learning of Chaotic Attractors , 1999, NIPS.

[6]  Eckmann,et al.  Liapunov exponents from time series. , 1986, Physical review. A, General physics.

[7]  J.C. Principe,et al.  Reconstructed dynamics and chaotic signal modeling , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[8]  Stuart F. Bockman.  Parameter Estimation for Chaotic Systems , 1991, 1991 American Control Conference.

[9]  C. L. Giles,et al.  Neural learning of chaotic dynamics: the error propagation algorithm , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[10]  Farmer,et al.  Predicting chaotic time series. , 1987, Physical review letters.

[11]  J. E. Jackson.  A User's Guide to Principal Components , 1991 .

[12]  J. Edward Jackson,et al.  A User's Guide to Principal Components. , 1991 .

[13]  Firdaus E. Udwadia,et al.  An efficient QR based method for the computation of Lyapunov exponents , 1997 .

[14]  Hermann Haken,et al.  At least one Lyapunov exponent vanishes if the trajectory of an attractor does not contain a fixed point , 1983 .

[15]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[16]  C. M. van den Bleek,et al.  Neural networks for prediction and control of chaotic fluidized bed hydrodynamics: a first step , 1997 .

[17]  Hava T. Siegelmann,et al.  Computational capabilities of recurrent NARX neural networks , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[18]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[19]  J. Príncipe,et al.  Local dynamic modeling with self-organizing maps and applications to nonlinear system identification and control , 1998, Proc. IEEE.

[20]  Gustavo Deco,et al.  Neural Learning of Chaotic System Behavior (Special Section on Nonlinear Theory and Its Applications) , 1994 .

[21]  Takens,et al.  Neural network model to control an experimental chaotic pendulum. , 1996, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[22]  H. J. T. Smith,et al.  Driven pendulum for studying chaos , 1989 .

[23]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[24]  A. Lapedes,et al.  Nonlinear signal processing using neural networks: Prediction and system modelling , 1987 .

[25]  A. Passamante,et al.  Using neural nets to look for chaos , 1992 .

[26]  S. Haykin,et al.  Chaotic dynamics of sea clutter. , 1999, Chaos.

[27]  Simon Haykin,et al.  Chaotic dynamics of sea clutter: an experimental study , 1997 .

[28]  Eric A. Wan,et al.  Time series prediction by using a connectionist network with internal delay lines , 1993 .

[29]  J. Yorke,et al.  Chaotic behavior of multidimensional difference equations , 1979 .

[30]  E. Ott.  Chaos in Dynamical Systems: Contents , 1993 .

[31]  C. Omlin,et al.  Prediction of seismic events in mines using neural networks , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[32]  P. Werbos,et al.  Long-term predictions of chemical processes using recurrent neural networks: a parallel training approach , 1992 .

[33]  Schouten,et al.  Experimental control of a chaotic pendulum with unknown dynamics using delay coordinates. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[34]  Peter Tiño,et al.  Learning long-term dependencies in NARX recurrent neural networks , 1996, IEEE Trans. Neural Networks.

[35]  Ioannis G. Kevrekidis,et al.  DISCRETE- vs. CONTINUOUS-TIME NONLINEAR SIGNAL PROCESSING OF Cu ELECTRODISSOLUTION DATA , 1992 .

[36]  Ioannis G. Kevrekidis,et al.  Model identification of a spatiotemporally varying catalytic reaction , 1993 .

[37]  Jose C. Principe,et al.  Prediction of Chaotic Time Series with Neural Networks , 1992 .

[38]  A. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[39]  Takens,et al.  Maximum-likelihood estimation of the entropy of an attractor. , 1994, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[40]  L. A. Aguirre,et al.  Validating Identified Nonlinear Models with Chaotic Dynamics , 1994 .

[41]  Neal B. Abraham,et al.  Lorenz-like chaos in NH3-FIR lasers , 1995 .

[42]  David S. Broomhead,et al.  Phase portraits from a time series: A singular system approach , 1987 .

[43]  Simon Haykin,et al.  Making sense of a complex world , 1998 .

[44]  Holger Kantz,et al.  Unbiased reconstruction of the dynamics underlying a noisy chaotic time series. , 1996, Chaos.

[45]  S. Haykin,et al.  Making sense of a complex world [chaotic events modeling] , 1998, IEEE Signal Process. Mag..

[46]  Takens,et al.  Estimation of the dimension of a noisy attractor. , 1994, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[47]  P. Grassberger,et al.  NONLINEAR TIME SEQUENCE ANALYSIS , 1991 .