Survival Density Particle Swarm Optimization for Neural Network Training

The Particle Swarm Optimizer (PSO) has previously been used to train neural networks and generally met with success. The advantage of the PSO over many of the other optimization algorithms is its relative simplicity and quick convergence. But those particles collapse so quickly that it exits a potentially dangerous property: stagnation, which state would make it impossible to arrive at the global optimum, even a local optimum. The ecological and physical universal laws enlighten us to improve the PSO algorithm. We introduce a concept, swarm’s survival density, into PSO for balancing the gravity and repulsion forces between two particles. A modified algorithm, survival density particle swarm optimization (SDPSO) is proposed for neural network training in this paper. Then it is applied to benchmark function minimization problems and neural network training for benchmark dataset classification problems. The experimental results illustrate its efficiency.

[1]  Robert E. Park,et al.  Physics and Society , 1940 .

[2]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[3]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[4]  F. van den Bergh,et al.  Training product unit networks using cooperative particle swarm optimisers , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[5]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[6]  Russell C. Eberhart,et al.  Comparison between Genetic Algorithms and Particle Swarm Optimization , 1998, Evolutionary Programming.

[7]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[8]  Peter J. Bentley,et al.  Don't push me! Collision-avoiding swarms , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[9]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[10]  James Kennedy,et al.  Bare bones particle swarms , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).