Optimizing Back-Propagation using PSO_Hill_A* and Genetic Algorithm

Back propagation(BP) is used to solve real world problems which use the concept of multilayer perceptron(MLP). BP have the disadvantage of trapped in local minima, slow convergence rate and more error prone. To optimize BP Algorithm, Particle swarm optimization(PSO) and Genetic algorithm(GA) is used. Limitation of PSO_Hill and PSO_A* is overcomes when these algorithms are combined and on the basis of strength of these two algorithm we proposed a new PSO_Hill_A* algorithm which is used to optimize and enhance learning process in terms of convergence rate and accuracy. GA is a kind of method to simulate and to search the optimal solution, GA can have four operations including Encoding, selecting, crossover and Mutation. To optimize and improve BP, we proposed two architecture: 1) Use of PSO_Hill_A* before and after hidden layer. 2) Use of GA before and after hidden layer.

[1]  H. Shayeghi,et al.  A HYBRID PARTICLE SWARM OPTIMIZATION BACK PROPAGATION ALGORITHM FOR SHORT TERM LOAD FORECASTING , 2011 .

[2]  Dipti Srinivasan,et al.  Evolving artificial neural networks for short term load forecasting , 1998, Neurocomputing.

[3]  A. Gray,et al.  I. THE ORIGIN OF SPECIES BY MEANS OF NATURAL SELECTION , 1963 .

[4]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[5]  Yan Meng,et al.  Reinforcement learning for neural networks using swarm intelligence , 2008, 2008 IEEE Swarm Intelligence Symposium.

[6]  V. Selvi,et al.  Comparative Analysis of Ant Colony and Particle Swarm Optimization Techniques , 2010 .

[7]  Gwo-Ching Liao,et al.  Application of a fuzzy neural network combined with a chaos genetic algorithm and simulated annealing to short-term load forecasting , 2006, IEEE Transactions on Evolutionary Computation.

[8]  Hossein Shayeghi,et al.  Intelligent Neural Network Based STLF , 2009 .

[9]  C. Darwin.  The Origin of Species by Means of Natural Selection, Or, The Preservation of Favoured Races in the Struggle for Life , 1859 .

[10]  Yiit Karpat,et al.  Swarm-intelligent neural network system (SINNS) based multi-objective optimization of hard turning , 2006 .

[11]  Michael R. Lyu,et al.  A hybrid particle swarm optimization-back-propagation algorithm for feedforward neural network training , 2007, Appl. Math. Comput..

[12]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[13]  Carlos Gershenson,et al.  Artificial Neural Networks for Beginners , 2003, ArXiv.

[14]  Chia-Feng Juang,et al.  A hybrid of genetic algorithm and particle swarm optimization for recurrent network design , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[16]  Priyanka Sharma,et al.  Optimizing BackPropagation using PSO _ Hill and PSO _ A * , 2013 .

[17]  Z.A. Bashir,et al.  Applying Wavelets to Short-Term Load Forecasting Using PSO-Based Neural Networks , 2009, IEEE Transactions on Power Systems.

[18]  Darrell Whitley,et al.  A genetic algorithm tutorial , 1994, Statistics and Computing.

[19]  Azzam ul Asar,et al.  Short Term Load Forecasting Using Particle Swarm Optimization Based ANN Approach , 2007, 2007 International Joint Conference on Neural Networks.

[20]  D. E. Goldberg,et al.  Simple Genetic Algorithms and the Minimal, Deceptive Problem , 1987 .

[21]  Hossein Shayeghi,et al.  STLF Based on Optimized Neural Network Using PSO , 2009 .

[22]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[23]  S. Pérez.  Apply genetic algorithm to the learning phase of a neural network , 2008 .