Learning persistent dynamics with neural networks

Persistence is one of the most common characteristics of real-world time series. In this work we investigate the process of learning persistent dynamics by neural networks. We show that for chaotic times series the network can get stuck for long training periods in a trivial minimum of the error function related to the long-term autocorrelation in the series. Remarkably, in these cases the transition to the trained phase is quite abrupt. For noisy dynamics the training process is smooth. We also consider the effectiveness of two of the most frequently used decorrelation methods in avoiding the problems related to persistence. Copyright 1997 Elsevier Science Ltd.

[1]  Anastasios A. Tsonis,et al.  Nonlinear Prediction, Chaos, and Noise. , 1992 .

[2]  H A Ceccatto,et al.  Predicting Indian monsoon rainfall: a neural network approach , 1994 .

[3]  H. Waelbroeck,et al.  Deterministic Chaos in Tropical Atmospheric Dynamics. , 1994, comp-gas/9410001.

[4]  David E. Rumelhart,et al.  Predicting the Future: a Connectionist Approach , 1990, Int. J. Neural Syst..

[5]  R. Calvo,et al.  Neural Network Prediction of Solar Activity , 1995 .

[6]  Fraser,et al.  Independent coordinates for strange attractors from mutual information. , 1986, Physical review. A, General physics.

[7]  H. D. Navone,et al.  Learning chaotic dynamics by neural networks , 1995 .

[8]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[9]  James B. Elsner,et al.  Predicting time series using a neural network as a method of distinguishing chaos from noise , 1992 .

[10]  D. Hathaway,et al.  The shape of the sunspot cycle , 1994 .

[11]  H A Ceccatto,et al.  Forecasting chaos from small data sets: a comparison of different nonlinear algorithms , 1995 .

[12]  C. E. Leith,et al.  Predictability of climate , 1978, Nature.

[13]  E. Lorenz.  Dimension of weather and climate attractors , 1991, Nature.

[14]  A. Refenes.  Neural Networks in the Capital Markets , 1994 .

[15]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[16]  Ping Chen,et al.  Empirical and theoretical evidence of economic chaos , 1988 .

[17]  P. Grassberger,et al.  Characterization of Strange Attractors , 1983 .