IMPROVING THE ACCURACY OF GRADIENT DESCENT BACK PROPAGATIONALGORITHM (GDAM) ON CLASSIFICATION PROBLEMS

The traditional Back-propagation Neural Network (BPNN) Algorithm is widely used in solving many real time problems in world. But BPNN possesses a problem of slow convergence and convergence to local minima. Previously, several modifications are suggested to improve the convergence rate of Gradient Descent Back-propagation algorithm such as careful selection of initial weights and biases, learning rate, momentum, network topology, activation function and ‘gain’ value in the activation function. This research proposed an algorithm for improving the current working performance of Back-propagation algorithm by adaptively changing the momentum value and at the same time keeping the ‘gain’ parameter fixed for all nodes in the neural network. The performance of the proposed method known as ‘Gradient Descent Method with Adaptive Momentum (GDAM)’ is compared with the performances of ‘Gradient Descent Method with Adaptive Gain (GDM-AG)’ and ‘Gradient Descent with Simple Momentum (GDM)’. The efficiency of the proposed method is demonstrated by simulations on five classification problems. Results show that GDAM can be used as an alternative approach for BPNN because it demonstrate better accuracy ratio on the chosen classification problems.

[1]  Richard S. Johannes,et al.  Using the ADAP Learning Algorithm to Forecast the Onset of Diabetes Mellitus , 1988 .

[2]  Nazri Mohd Nawi.  Computational issues in process optimisation using historical data , 2007 .

[3]  Kaspar Althoefer,et al.  Stability analysis of a three-term backpropagation algorithm , 2005, Neural Networks.

[4]  Shen Zhang,et al.  Improved BP Neural Network for Transformer Fault Diagnosis , 2007 .

[5]  O. Mangasarian,et al.  Multisurface method of pattern separation for medical diagnosis applied to breast cytology. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[6]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[7]  J. Ross Quinlan,et al.  Simplifying decision trees , 1987, Int. J. Hum. Comput. Stud..

[8]  R. Detrano,et al.  International application of a new probability algorithm for the diagnosis of coronary artery disease. , 1989, The American journal of cardiology.

[9]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[10]  Petr Musilek,et al.  CHAPTER 7 – Neural Networks and Fuzzy Systems , 2000 .

[11]  John Mark Bishop,et al.  Simple adaptive momentum: New algorithm for training multilayer perceptrons , 1994 .

[12]  R.J. Mitchell,et al.  On Simple Adaptive Momentum , 2008, 2008 7th IEEE International Conference on Cybernetic Intelligent Systems.

[13]  Rozaida Ghazali,et al.  The Effect of Adaptive Gain and Adaptive Momentum in Improving Training Time of Gradient Descent Back Propagation Algorithm on Classification Problems , 2011 .

[14]  Vladimir M. Krasnopolsky,et al.  Some neural network applications in environmental sciences. Part II: advancing computational efficiency of environmental numerical models , 2003, Neural Networks.

[15]  Ben Coppin,et al.  Artificial Intelligence Illuminated , 2004 .

[16]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[17]  David E. Booth,et al.  A comparison of supervised and unsupervised neural networks in predicting bankruptcy of Korean firms , 2005, Expert Syst. Appl..

[18]  Nazri Mohd Nawi,et al.  Noise-Induced Hearing Loss (NIHL) Prediction in Humans Using a Modified Back Propagation Neural Network , 2011 .

[19]  Nazri Mohd Nawi,et al.  An Improved Conjugate Gradient Based Learning Algorithm for Back Propagation Neural Networks , 2008 .

[20]  Hongmei Shao,et al.  A New BP Algorithm with Adaptive Momentum for FNNs Training , 2009, 2009 WRI Global Congress on Intelligent Systems.

[21]  I A Basheer,et al.  Artificial neural networks: fundamentals, computing, design, and application. , 2000, Journal of microbiological methods.

[22]  M. A. Fkirin,et al.  Change detection using neural network in Toshka area , 2009, 2009 National Radio Science Conference.