Particles Gradient: A New Approach to Perform MLP Neural Networks Training based on Particles Swarm Optimization

The use of heuristic algorithms in neural networks training is not a new subject. Several works have already accomplished good results, however not competitive with procedural methods for problems where the gradient of the error is well defined. The present document proposes an alternative for neural networks training using PSO(Particle Swarm Optimization) to evolve the training process itself and not to evolve directly the network parameters as usually. This way we get quite superior results and obtain a method clearly faster than others known methods for training neural networks using heuristic algorithms.

[1]  J. Urgen Branke.  Evolutionary Algorithms for Neural Network Design and Training , 1995 .

[2]  Lawrence Davis,et al.  Training Feedforward Neural Networks Using Genetic Algorithms , 1989, IJCAI.

[3]  Yves Chauvin,et al.  Backpropagation: the basic theory , 1995 .

[4]  Martin Fodslette Møller,et al.  Learning by Conjugate Gradients , 1990, IMYCS.

[5]  Udo Seiffert,et al.  Multiple Layer Perceptron training using genetic algorithms , 2001, ESANN.

[6]  E. Fernández,et al.  Finding Optimal Neural Network Architecture Using Genetic Algorithms , 2007 .

[7]  Amreen Khan,et al.  An Analysis of Particle Swarm Optimization with Data Clustering-Technique for Optimization in Data Mining. , 2010 .

[8]  Craig W. Reynolds.  Flocks, herds, and schools: a distributed behavioral model , 1987, SIGGRAPH.

[9]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[10]  Colin Fyfe.  Artificial Neural Networks , 2005 .

[11]  David J. Chalmers,et al.  The Evolution of Learning: An Experiment in Genetic Connectionism , 1991 .

[12]  David G. Luenberger,et al.  Linear and Nonlinear Programming: Second Edition , 2003 .

[13]  Roberto Battiti,et al.  BFGS Optimization for Faster and Automated Supervised Learning , 1990 .

[14]  Ulf Grenander,et al.  A stochastic nonlinear model for coordinated bird flocks , 1990 .

[15]  Martin Fodslette Meiller.  A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning , 1993 .

[16]  Jianxin Wu,et al.  Genetic Algorithm based Selective Neural Network Ensemble , 2001, IJCAI.

[17]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .