Back-Propagation vs Particle Swarm Optimization Algorithm: which Algorithm is better to adjust the Synaptic Weights of a Feed-Forward ANN?

Bio-inspired algorithms have shown their usefulness in different non-linear opti-mization problems. Due to their efficiency and adaptability, these algorithms have been applied to a wide range of problems. In this paper we compare two ways of training an artificial neural network (ANN): Particle Swarm Optimization (PSO) algorithms against classical training algorithms such as: back-propagation (BP) and Levenberg Marquardt method. The main contribution of this paper is to answer the next question: is PSO really better than classical training algorithms in adjusting the synaptic weights of an ANN? First of all, we explain how the ANN training phase could be seen as an optimization problem. Then, it is explained how PSO could be applied to find the best synaptic weights of the ANN. Finally, we perform a comparison among different classical methods and PSO approach when an ANN is applied to different non-linear problems and to a real object recognition problem.

[1]  Lifeng Xi,et al.  An Improved Particle Swarm Optimization for Evolving Feedforward Artificial Neural Networks , 2007, Neural Processing Letters.

[2]  Cheng-Yan Kao,et al.  A Robust Evolutionary Algorithm for Training Neural Networks , 2001, Neural Computing & Applications.

[3]  Kwok-wing Chau,et al.  Application of a PSO-based neural network in analysis of outcomes of construction claims , 2007 .

[4]  Juan Humberto Sossa Azuela,et al.  Design of artificial neural networks using a modified Particle Swarm Optimization algorithm , 2009, 2009 International Joint Conference on Neural Networks.

[5]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[6]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[7]  Ge Xiurun,et al.  An improved PSO-based ANN with simulated annealing technique , 2005, Neurocomputing.

[8]  Kiyotaka Izumi,et al.  A particle-swarm-optimized fuzzy-neural network for voice-controlled robot systems , 2005, IEEE Transactions on Industrial Electronics.

[9]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[10]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[11]  Liang Zhao,et al.  PSO-based single multiplicative neuron model for time series prediction , 2009, Expert Syst. Appl..

[12]  Patrick van der Smagt,et al.  Introduction to neural networks , 1995, The Lancet.

[13]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[14]  Juan Julián Merelo Guervós,et al.  Evolving Multilayer Perceptrons , 2000, Neural Processing Letters.

[15]  Zhiwei Wang,et al.  Particle swarm optimization and neural network application for QSAR , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..