CSLM: Levenberg Marquardt based Back Propagation Algorithm Optimized with Cuckoo Search

Training an artificial neural network is an optimization task, since it is desired to find optimal weight sets for a neural network during training process. Traditional training algorithms such as back propagation have some drawbacks such as getting stuck in local minima and slow speed of convergence. This study combines the best features of two algorithms; i.e. Levenberg Marquardt back propagation (LMBP) and Cuckoo Search (CS) for improving the convergence speed of artificial neural networks (ANN) training. The proposed CSLM algorithm is trained on XOR and OR datasets. The experimental results show that the proposed CSLM algorithm has better performance than other similar hybrid variants used in this study.

[1]  Zhen-Ya He,et al.  Neural network and its application on machinery fault diagnosis , 1992, [Proceedings 1992] IEEE International Conference on Systems Engineering.

[2]  M. Z. Rehman,et al.  IMPROVING THE ACCURACY OF GRADIENT DESCENT BACK PROPAGATIONALGORITHM (GDAM) ON CLASSIFICATION PROBLEMS , 2011 .

[3]  Cao Xiao-ping.  Fault prediction for inertial device based on LMBP neural network , 2005 .

[4]  Ben Coppin,et al.  Artificial Intelligence Illuminated , 2004 .

[5]  Jun YAN,et al.  Levenberg-Marquardt Algorithm Applied to Forecast the Ice Conditions in Ningmeng Reach of the Yellow River , 2009, 2009 Fifth International Conference on Natural Computation.

[6]  W.A.M. Ahmed,et al.  Modified back propagation algorithm for learning artificial neural networks , 2001, Proceedings of the Eighteenth National Radio Science Conference. NRSC'2001 (IEEE Cat. No.01EX462).

[7]  Xin-She Yang,et al.  Cuckoo Search via Lévy flights , 2009, 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC).

[8]  Wen Jin,et al.  The improvements of BP neural network learning algorithm , 2000, WCC 2000 - ICSP 2000. 2000 5th International Conference on Signal Processing Proceedings. 16th World Computer Congress 2000.

[9]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[10]  T.,et al.  Training Feedforward Networks with the Marquardt Algorithm , 2004 .

[11]  I. Shereef,et al.  A New Weather Forecasting Technique using Back Propagation Neural Network with Modified Levenberg-Marquardt Algorithm for Learning , 2011 .

[12]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[13]  Randall S. Sexton,et al.  Toward global optimization of neural networks: A comparison of the genetic algorithm and backpropagation , 1998, Decis. Support Syst..

[14]  Simon Haykin,et al.  Neural networks , 1994 .

[15]  Nazri Mohd Nawi,et al.  BPGD-AG: A New Improvement Of Back-Propagation Neural Network Learning Algorithms With Adaptive Gain , 2010 .

[16]  Y. Radhika,et al.  Atmospheric Temperature Prediction using Support Vector Machines , 2009 .

[17]  Nazri Mohd Nawi,et al.  The 4 th International Conference on Electrical Engineering and Informatics ( ICEEI 2013 ) A New Levenberg Marquardt Based Back Propagation Algorithm Trained with Cuckoo Search , 2013 .

[18]  I A Basheer,et al.  Artificial neural networks: fundamentals, computing, design, and application. , 2000, Journal of microbiological methods.

[19]  Vladimir M. Krasnopolsky,et al.  Some neural network applications in environmental sciences. Part II: advancing computational efficiency of environmental numerical models , 2003, Neural Networks.

[20]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[21]  E. Mine Cinar,et al.  Neural Networks: A New Tool for Predicting Thrift Failures , 1992 .

[22]  Ehsan Mesbahi,et al.  Artificial neural networks: fundamentals , 2003 .

[23]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[24]  B.M. Wilamowski,et al.  Neural Network Trainer with Second Order Learning Algorithms , 2007, 2007 11th International Conference on Intelligent Engineering Systems.

[25]  Can Çinar,et al.  Artificial neural network based modeling of heated catalytic converter performance , 2005 .

[26]  Salim Lahmiri,et al.  A Comparative Study Of Backpropagation Algorithms In Financial Prediction , 2011 .