Online designed of Echo State Network based on Particle Swarm Optimization for system identification

Complexities with existing algorithms have thus far limited supervised training techniques for Recurrent Neural Networks (RNNs) from widespread use. Echo State Network (ESN) presents a novel approach to train RNNs. Certain properties make ESN online learning unsuitable. This paper proposes a modified version of ESN structure for complex nonlinear system online prediction. The Particle Swarm Optimization (PSO) is adopted to online train the output weights of ESN, as against computing it, which greatly improve the modeling accuracy, avoid derivative calculations, and expand the scope of application. The nonlinear system, static function SinC and Mackey-Glass chaos mapping are used to verify the effectiveness of the proposed ESN+PSO approach.

[1]  Gowrishankar,et al.  Neural Network Based Traffic Prediction for Wireless Data Networks , 2008, Int. J. Comput. Intell. Syst..

[2]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[3]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[4]  John G. Harris,et al.  Noise-Robust Automatic Speech Recognition Using a Predictive Echo State Network , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[5]  Xingjuan Cai.  Using Small-World Model to Improve the Performance of Alignment Particle Swarm Optimization , 2009, 2009 Ninth International Conference on Hybrid Intelligent Systems.

[6]  Günther Palm,et al.  A neural framework for adaptive robot control , 2009, Neural Computing and Applications.

[7]  Garrison W. Cottrell,et al.  2007 Special Issue: Learning grammatical structure with Echo State Networks , 2007 .

[8]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[9]  Ganesh K. Venayagamoorthy,et al.  Online design of an echo state network based wide area monitor for a multimachine power system , 2007, Neural Networks.

[10]  P. J. Angeline,et al.  Using selection to improve particle swarm optimization , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[11]  Yung C. Shin,et al.  Constructive training of recurrent neural networks using hybrid optimization , 2010, Neurocomputing.

[12]  R.G. Harley,et al.  Utilization of Echo State Networks for Differentiating Source and Nonlinear Load Harmonics in the Utility Network , 2008, IEEE Transactions on Power Electronics.

[13]  Robert K. L. Gay,et al.  Error Minimized Extreme Learning Machine With Growth of Hidden Nodes and Incremental Learning , 2009, IEEE Transactions on Neural Networks.

[14]  Song Qing-song,et al.  Multiple clusters echo state network for chaotic time series prediction , 2009 .

[15]  Amparo Alonso-Betanzos,et al.  Linear-least-squares initialization of multilayer perceptrons through backpropagation of the desired response , 2005, IEEE Transactions on Neural Networks.

[16]  Debashis Chatterjee,et al.  A PSO based optimal switching technique for voltage harmonic reduction of multilevel inverter , 2010, Expert Syst. Appl..

[17]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.