An approach to reservoir computing design and training

Reservoir computing is a framework for computation like a recurrent neural network that allows for the black box modeling of dynamical systems. In contrast to other recurrent neural network approaches, reservoir computing does not train the input and internal weights of the network, only the readout is trained. However it is necessary to adjust parameters to create a ''good'' reservoir for a given application. In this study we introduce a method, called RCDESIGN (reservoir computing and design training). RCDESIGN combines an evolutionary algorithm with reservoir computing and simultaneously looks for the best values of parameters, topology and weight matrices without rescaling the reservoir matrix by the spectral radius. The idea of adjust the spectral radius within the unit circle in the complex plane comes from the linear system theory. However, this argument does not necessarily apply to nonlinear systems, which is the case of reservoir computing. The results obtained with the proposed method are compared with results obtained by a genetic algorithm search for global parameters generation of reservoir computing. Four time series were used to validate RCDESIGN.

[1]  Benjamin Schrauwen,et al.  An experimental unification of reservoir computing methods , 2007, Neural Networks.

[2]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[3]  Teresa Bernarda Ludermir,et al.  Using Reservoir Computing for Forecasting Time Series: Brazilian Case Study , 2008, 2008 Eighth International Conference on Hybrid Intelligent Systems.

[4]  Peter Ford Dominey.  Complex sensory-motor sequence learning based on recurrent state representation and reinforcement learning , 1995, Biological Cybernetics.

[5]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[6]  Milde M. S. Lira,et al.  Recurrent neural networks solving a real large scale mid-term scheduling for power plants , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[7]  Benjamin Liebald,et al.  Exploration of effects of different network topologies on the ESN signal crosscorrelation matrix spectrum , 2004 .

[8]  Benjamin Schrauwen,et al.  On the Quantification of Dynamics in Reservoir Computing , 2009, ICANN.

[9]  J.J. Steil,et al.  Backpropagation-decorrelation: online recurrent learning with O(N) complexity , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[10]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[11]  S. Walson.  Fresh forecasts [wind power forecasting] , 2005 .

[12]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[13]  James E. Baker,et al.  Reducing Bias and Inefficienry in the Selection Algorithm , 1987, ICGA.

[14]  S. Quartz,et al.  Getting to Know You: Reputation and Trust in a Two-Person Economic Exchange , 2005, Science.

[15]  Jack Dongarra,et al.  LAPACK User's Guide / E. Anderson ... , 1999 .

[16]  Jack Dongarra,et al.  LAPACK Users' Guide, 3rd ed. , 1999 .

[17]  T. van der Zant,et al.  Identification of motion with echo state network , 2004, Oceans '04 MTS/IEEE Techno-Ocean '04 (IEEE Cat. No.04CH37600).

[18]  José Carlos Príncipe,et al.  Analysis and Design of Echo State Networks , 2007, Neural Computation.

[19]  Benjamin Schrauwen,et al.  Event detection and localization for small mobile robots using reservoir computing , 2008, Neural Networks.

[20]  Teresa Bernarda Ludermir,et al.  Genetic algorithm for reservoir computing optimization , 2009, 2009 International Joint Conference on Neural Networks.

[21]  Teresa Bernarda Ludermir,et al.  Investigating the use of Reservoir Computing for forecasting the hourly wind speed in short -term , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[22]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[23]  Benjamin Schrauwen,et al.  The Introduction of Time-Scales in Reservoir Computing, Applied to Isolated Digits Recognition , 2007, ICANN.

[24]  M. C. Ozturk,et al.  Computing with transiently stable states , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[25]  Lutz Prechelt,et al.  A Set of Neural Network Benchmark Problems and Benchmarking Rules , 1994 .

[26]  Herbert Jaeger,et al.  Adaptive Nonlinear System Identification with Echo State Networks , 2002, NIPS.

[27]  Teresa Bernarda Ludermir,et al.  Evolutionary strategy for simultaneous optimization of parameters, topology and reservoir weights in Echo State Networks , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).