Structure optimization of dynamic reservoir ensemble using genetic algorithm

Reservoir computing has been widely applied in dynamical system modeling and solving time-dependent problems at low computational expense. However, when confronting some complex tasks that exhibit multiple sets of dynamics, the conventional reservoir computing model with a single reservoir may become ineffective and powerless. Inspired by the modality-independent but functionally connected brain regions, the concept of reservoir ensemble has been proposed which contains multiple reservoirs. In this paper, we propose a new dynamic reservoir ensemble model which is capable of automatically adapting and optimizing the synaptic and structural plasticity of a reservoir ensemble towards an optimal performance using the genetic algorithm. As shown in a real-life time series application — temperature prediction, the proposed model demonstrates superior performance over both the conventional single-reservoir model and the static reservoir ensemble model.

[1]  Herbert Jaeger,et al.  Adaptive Nonlinear System Identification with Echo State Networks , 2002, NIPS.

[2]  David B. Fogel,et al.  An introduction to simulated evolutionary optimization , 1994, IEEE Trans. Neural Networks.

[3]  Yan Meng,et al.  Reservoir computing ensembles for multi-object behavior recognition , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[4]  Jürgen Schmidhuber,et al.  Training Recurrent Networks by Evolino , 2007, Neural Computation.

[5]  Danyang Li,et al.  An ensemble convolutional echo state networks for facial expression recognition , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[6]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[7]  Jean-Pierre Martens,et al.  Acoustic Modeling With Hierarchical Reservoirs , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[8]  Benjamin Schrauwen,et al.  Phoneme Recognition with Large Hierarchical Reservoirs , 2010, NIPS.

[9]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[10]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[11]  Jürgen Schmidhuber,et al.  Modeling systems with internal state using evolino , 2005, GECCO '05.

[12]  Simon Haykin,et al.  Decoupled echo state networks with lateral inhibition , 2007, Neural Networks.

[13]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[14]  Paul-Gerhard Plöger,et al.  Echo State Networks for Mobile Robot Modeling and Control , 2003, RoboCup.

[15]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[16]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[17]  Kevin Curran,et al.  An experimental evaluation of echo state network for colour image segmentation , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[18]  S. Schultz.  Principles of Neural Science, 4th ed. , 2001 .

[19]  Abdelkader Benyettou,et al.  Novel Approach Using Echo State Networks for Microscopic Cellular Image Segmentation , 2016, Cognitive Computation.