An improved particle swarm optimization based training algorithm for neural network

The Particle Swarm Optimization (PSO) method was originally designed by Kennedy and Eberhart in 1995 and has been applied successfully to various optimization problems. The PSO idea is inspired by natural concepts such as fish schooling, bird flocking and human social relations. Backpropagation (BP) is generally used for neural network training. It is very important to choose a proper algorithm for training a neural network. In this paper, we present a modified particle swarm optimization based training algorithm for neural network. The proposed method modify the trajectories (positions and velocities) of the particle based on the best positions visited earlier by themselves and other particles, and also incorporates population diversity method to avoid premature convergence. Experimental results have demonstrated that the modified PSO is a useful tool for training neural network.

[1]  Puyin Liu.  An Improved Feedforward Fuzzy Neural Network and Its Learning Algorithm , 2004, J. Multiple Valued Log. Soft Comput..

[2]  Shi Meng.  Nonlinear Dynamic Modelling of Sensors Based on Recursive Neural Network , 2004 .

[3]  Yuhui Shi,et al.  Particle swarm optimization: developments, applications and resources , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[4]  Shi Jian Zhao,et al.  Levenberg-Marquardt algorithm for nonlinear principal component analysis neural network through inputs training , 2004, Fifth World Congress on Intelligent Control and Automation (IEEE Cat. No.04EX788).

[5]  Hongxing Li,et al.  Efficient learning algorithms for three-layer regular feedforward fuzzy neural networks , 2004, IEEE Trans. Neural Networks.

[6]  Hirotaka Yoshida,et al.  A particle swarm optimization for reactive power and voltage control in electric power systems considering voltage security assessment , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[7]  Xianhua Dai,et al.  Cumulant-based training algorithms of two-layer feedforward neural networks , 2000, Signal Process..

[8]  Hong-Xing Li,et al.  INTERPOLATION FUNCTIONS OF FEEDFORWARD NEURAL NETWORKS , 2003 .

[9]  何小荣,et al.  A Method for Solving Computer-Aided Product Design Optimization Problem Based on Back Propagation Neural Network , 2004 .

[10]  Kazuyuki Aihara,et al.  Back-propagation learning of infinite-dimensional dynamical systems , 1997, Neural Networks.

[11]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[12]  MATT SETTLES,et al.  Neural Network Learning using Particle Swarm Optimizers , 2002 .