Optimum weights and biases for feed forward neural network by particle swarm optimisation

This paper introduces particle swarm intelligence (PSI) in feed forward neural network (FFNN) with backpropagation for finding initial weights and biases of the feed forward neural network. The combination of particle swarm optimisation (PSO) and FFNN greatly help in fast convergence of FFNN in classification and prediction to various benchmark problems by overcoming the disadvantage of backpropagation of getting stuck at local minima or local maxima. The benchmarking databases for neural network contain various datasets from various different domains. All datasets represent realistic problems which could be called diagnosis tasks and all but one consist of real world data. Two such benchmarking problems are selected in this paper for comparison and the performance of PSO with FFNN for finding weights and biases is implemented and compared with random initialisation of weights and biases with normal FFNN. The result shows that using PSO minimises the prediction error.

[1]  Mario Ventresca,et al.  Improving the Convergence of Backpropagation by Opposite Transfer Functions , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[2]  Zhang Dingxue,et al.  A Modified Particle Swarm Optimization with an Adaptive Acceleration Coefficients , 2009, 2009 Asia-Pacific Conference on Information Processing.

[3]  Michael N. Vrahatis,et al.  On the computation of all global minimizers through particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[4]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[5]  Zbigniew Michalewicz,et al.  Handbook of Evolutionary Computation , 1997 .

[6]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[7]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[8]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[9]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[10]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[11]  Bidyadhar Subudhi,et al.  Differential Evolution and Levenberg Marquardt Trained Neural Network Scheme for Nonlinear System Identification , 2008, Neural Processing Letters.

[12]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.