Parameter Optimization in a Leaky Integrator Echo State Network with an Improved Gravitational Search Algorithm

In the prediction of a nonlinear time series based on a leaky integrator echo state network (leaky-ESN), building a reservoir related to the specific problem is a key step. For problems such as poor performance of randomly generated reservoirs, it is tough to determine the parameter values of the reservoirs. The work in this paper uses the gravitational search algorithm (GSA) to optimize the global parameters of a leaky-ESN, such as the leaking rate, the spectral radius, and the input scaling factor. The basic GSA has some problems, such as slow convergence and poor balance between exploration and exploitation, and it cannot solve some complex optimization problems well. To solve these problems, an improved gravitational search algorithm (IGSA) is proposed in this paper. First, the best agent and elite agents were archived and utilized to accelerate the exploration phase and improve the convergence rate in the exploitation phase. Second, to improve the effect of the poor fitness agents on the optimization result, a differential mutation strategy was proposed, which generated new individuals to replace original agents with worse fitness, increasing the diversity of the population and improving the global optimization ability of the algorithm. Finally, two simulation experiments showed that the leaky-ESN optimized by the IGSA had better prediction accuracy.

[1]  Fengrong Bi,et al.  An Engine Fault Detection Method Based on the Deep Echo State Network and Improved Multi-Verse Optimizer , 2022, Energies.

[2]  Yinggao Yue,et al.  A Novel Coverage Optimization Strategy Based on Grey Wolf Algorithm Optimized by Simulated Annealing for Wireless Sensor Networks , 2021, Comput. Intell. Neurosci..

[3]  Mingxuan Sun,et al.  Echo State Network-Based Backstepping Adaptive Iterative Learning Control for Strict-Feedback Systems: An Error-Tracking Approach , 2020, IEEE Transactions on Cybernetics.

[4]  Brian R. King,et al.  Time series prediction using deep echo state networks , 2020, Neural Computing and Applications.

[5]  Xiao Liu,et al.  Trajectory Design and Power Control for Multi-UAV Assisted Wireless Networks: A Machine Learning Approach , 2018, IEEE Transactions on Vehicular Technology.

[6]  Lin Lin,et al.  Genetic algorithm optimized double-reservoir echo state network for multi-regime time series prediction , 2017, Neurocomputing.

[7]  Sarah L. Taylor,et al.  Optimizing Echo State Networks for Static Pattern Recognition , 2017, Cognitive Computation.

[8]  Xuefeng Yan,et al.  Optimizing the echo state network with a binary particle swarm optimization algorithm , 2015, Knowl. Based Syst..

[9]  Jie-Sheng Wang,et al.  Echo state networks based predictive model of vinyl chloride monomer convention velocity optimized by artificial fish swarm algorithm , 2014, Soft Comput..

[10]  Hossein Nezamabadi-pour,et al.  GSA: A Gravitational Search Algorithm , 2009, Inf. Sci..

[11]  D. Popovici,et al.  Optimization and applications of echo state networks with leaky- integrator neurons , 2007, Neural Networks.

[12]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[13]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[14]  Yong Xiao,et al.  Long-Short Term Echo State Network for Time Series Prediction , 2020, IEEE Access.

[15]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .