The 4 th International Conference on Electrical Engineering and Informatics ( ICEEI 2013 ) A New Levenberg Marquardt Based Back Propagation Algorithm Trained with Cuckoo Search

Back propagation training algorithm is widely used techniques in artificial neural network and is also very popular optimization task in finding an optimal weight sets during the training process. However, traditional back propagation algorithms have some drawbacks such as getting stuck in local minimum and slow speed of convergence. This research proposed an improved Levenberg Marquardt (LM) based back propagation (BP) trained with Cuckoo search algorithm for fast and improved convergence speed of the hybrid neural networks learning method. The performance of the proposed algorithm is compared with Artificial Bee Colony (ABC) and the other hybridized procedure of its kind. The simulation outcomes show that the proposed algorithm performed better than other algorithm used in this study in term of convergence speed and rate.

[1]  Mohamed Najim,et al.  A fast feedforward training algorithm using a modified form of the standard backpropagation algorithm , 2001, IEEE Trans. Neural Networks.

[2]  Yang Wei,et al.  Improved LMBP algorithm in the analysis and application of simulation data , 2010, 2010 International Conference on Computer Application and System Modeling (ICCASM 2010).

[3]  Xin Yao,et al.  Evolutionary Artificial Neural Networks , 1993, Int. J. Neural Syst..

[4]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[5]  Xin-She Yang,et al.  Engineering optimisation by cuckoo search , 2010 .

[6]  Rozaida Ghazali,et al.  Predicting patients with heart disease by using an improved back-propagation algorithm , 2011 .

[7]  Randall S. Sexton,et al.  Comparing backpropagation with a genetic algorithm for neural network training , 1999 .

[8]  Paulo Cortez,et al.  Particle swarms for feedforward neural network training , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[9]  E. Mine Cinar,et al.  Neural Networks: A New Tool for Predicting Thrift Failures , 1992 .

[10]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[11]  T.,et al.  Training Feedforward Networks with the Marquardt Algorithm , 2004 .

[12]  Jun YAN,et al.  Levenberg-Marquardt Algorithm Applied to Forecast the Ice Conditions in Ningmeng Reach of the Yellow River , 2009, 2009 Fifth International Conference on Natural Computation.

[13]  Xin-She Yang,et al.  Cuckoo Search via Lévy flights , 2009, 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC).

[14]  Judith E. Dayhoff,et al.  Neural Network Architectures: An Introduction , 1989 .

[15]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[16]  B.M. Wilamowski,et al.  Neural Network Trainer with Second Order Learning Algorithms , 2007, 2007 11th International Conference on Intelligent Engineering Systems.

[17]  Randall S. Sexton,et al.  Toward global optimization of neural networks: A comparison of the genetic algorithm and backpropagation , 1998, Decis. Support Syst..

[18]  Dervis Karaboga,et al.  Hybrid Artificial Bee Colony algorithm for neural network training , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[19]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[20]  Cao Xiao-ping.  Fault prediction for inertial device based on LMBP neural network , 2005 .

[21]  Xinghuo Yu,et al.  A general backpropagation algorithm for feedforward neural networks learning , 2002, IEEE Trans. Neural Networks.

[22]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[23]  S. Das.  Elements Of Artificial Neural Networks [Book Reviews] , 1998, IEEE Transactions on Neural Networks.

[24]  Tsung-Lin Lee,et al.  Back-propagation neural network for the prediction of the short-term storm surge in Taichung harbor, Taiwan , 2008, Eng. Appl. Artif. Intell..