Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication

We present a method for learning nonlinear systems, echo state networks (ESNs). ESNs employ artificial recurrent neural networks in a way that has recently been proposed independently as a learning mechanism in biological brains. The learning method is computationally efficient and easy to use. On a benchmark task of predicting a chaotic time series, accuracy is improved by a factor of 2400 over previous techniques. The potential for engineering applications is illustrated by equalizing a communication channel, where the signal error rate is improved by two orders of magnitude.

[1]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[2]  S Dehaene,et al.  Neural networks that learn temporal sequences by selection. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[4]  Sheng Chen,et al.  Proceedings SPIE - Advanced Signal Processing Algorithms, Architectures and Implementations , 1990 .

[5]  Yuichi Nakamura,et al.  Approximation of dynamical systems by continuous time recurrent neural networks , 1993, Neural Networks.

[6]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[7]  Xin Yao,et al.  A new evolutionary system for evolving artificial neural networks , 1997, IEEE Trans. Neural Networks.

[8]  J. Houk,et al.  Model of cortical-basal ganglionic processing: encoding the serial order of sensory events. , 1998, Journal of neurophysiology.

[9]  Johan A. K. Suykens,et al.  Nonlinear modeling : advanced black-box techniques , 1998 .

[10]  G B Stanley,et al.  Reconstruction of Natural Scenes from Ensemble Responses in the Lateral Geniculate Nucleus , 1999, The Journal of Neuroscience.

[11]  Johan A. K. Suykens,et al.  WINNING ENTRY OF THE K. U. LEUVEN TIME-SERIES PREDICTION COMPETITION , 1999 .

[12]  Jerald D. Kralik,et al.  Real-time prediction of hand trajectory by ensembles of cortical neurons in primates , 2000, Nature.

[13]  Jürgen Schmidhuber,et al.  Applying LSTM to Time Series Predictable through Time-Window Approaches , 2000, ICANN.

[14]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[15]  Chris I. De Zeeuw,et al.  Dynamical Working Memory and Timed Responses: The Role of Reverberating Loops in the Olivo-Cerebellar System , 2002, Neural Computation.

[16]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[17]  C. Ruan,et al.  Ultrafast Electron Crystallography of Interfacial Water , 2004, Science.