Effect of hybrid circle reservoir injected with wavelet-neurons on performance of echo state network

The Echo State Network (ESN) has attracted wide attention for its superior performance in chaos time-series prediction. However, the complicated ESN topologies and the random reservoirs are difficult to implement in practice. We propose a hybrid circle reservoir (HCR) ESN architecture that comprises the following features: (1) built with low complexity circle reservoir; (2) partly injected with wavelet-neurons; (3) uses fixed connection weights in both input matrix and dynamic reservoir matrix. The HCR model has been successfully applied to solve six application problems, and the results are used to compare with the existing low complexity simple circle reservoir (SCR) ESN. Furthermore, we analyze the performance of the new model under different ratios of wavelet-neurons, different circle distributions and different input sign patterns. Simulation results show that the HCR model achieves significantly better performance in prediction accuracy than the SCR model. Additionally, the HCR model has similar low complexity as the SCR. Moreover, the short-term memory capacity (MC) in the HCR is close to the theoretical optimal MC value.

[1]  Amir F. Atiya,et al.  New results on recurrent network training: unifying the algorithms and accelerating convergence , 2000, IEEE Trans. Neural Networks Learn. Syst..

[2]  Yagyensh C. Pati,et al.  Analysis and synthesis of feedforward neural networks using discrete affine wavelet transformations , 1993, IEEE Trans. Neural Networks.

[3]  Zehong Yang,et al.  Short-term stock price prediction based on echo state networks , 2009, Expert Syst. Appl..

[4]  Herbert Jaeger,et al.  Optimization and applications of echo state networks with leaky- integrator neurons , 2007, Neural Networks.

[5]  Qinghua Zhang,et al.  Wavelet networks , 1992, IEEE Trans. Neural Networks.

[6]  Mantas Lukosevicius,et al.  A Practical Guide to Applying Echo State Networks , 2012, Neural Networks: Tricks of the Trade.

[7]  Qinghua Zhang,et al.  Using wavelet network in nonparametric estimation , 1997, IEEE Trans. Neural Networks.

[8]  F. AtiyaA.,et al.  New results on recurrent network training , 2000 .

[9]  Peter Tiño,et al.  Minimum Complexity Echo State Network , 2011, IEEE Transactions on Neural Networks.

[10]  Robert J. Schalkoff,et al.  Artificial neural networks , 1997 .

[11]  Herbert Jaeger,et al.  Adaptive Nonlinear System Identification with Echo State Networks , 2002, NIPS.

[12]  Simon Haykin,et al.  Decoupled echo state networks with lateral inhibition , 2007, Neural Networks.

[13]  Garrison W. Cottrell,et al.  2007 Special Issue: Learning grammatical structure with Echo State Networks , 2007 .

[14]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[15]  Teresa Orlowska-Kowalska,et al.  Adaptive Sliding-Mode Neuro-Fuzzy Control of the Two-Mass Induction Motor Drive Without Mechanical Sensors , 2010, IEEE Transactions on Industrial Electronics.

[16]  Cheng-jian Wei,et al.  Harnessing Non-linearity by Sigmoid-wavelet Hybrid Echo State Networks (SWHESN) , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[17]  John G. Harris,et al.  Noise-Robust Automatic Speech Recognition Using a Predictive Echo State Network , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[18]  N. Pindoriya,et al.  An Adaptive Wavelet Neural Network-Based Energy Price Forecasting in Electricity Markets , 2008, IEEE Transactions on Power Systems.

[19]  H. Jaeger,et al.  Reservoir riddles: suggestions for echo state network research , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[20]  Chi-Huang Lu,et al.  Wavelet Fuzzy Neural Networks for Identification and Predictive Control of Dynamic Systems , 2011, IEEE Transactions on Industrial Electronics.

[21]  Marc W. Slutzky,et al.  Manipulating epileptiform bursting in the rat hippocampus using chaos control and adaptive techniques , 2003, IEEE Transactions on Biomedical Engineering.

[22]  Lixiang Li,et al.  The architecture of dynamic reservoir in the echo state network. , 2012, Chaos.

[23]  M. Moraud Wavelet Networks , 2018, Foundations of Wavelet Networks and Applications.

[24]  M. Hénon,et al.  A two-dimensional mapping with a strange attractor , 1976 .

[25]  Zhidong Deng,et al.  Collective Behavior of a Small-World Recurrent Neural System With Scale-Free Distribution , 2007, IEEE Transactions on Neural Networks.

[26]  M. Hénon A two-dimensional mapping with a strange attractor , 1976 .