DETERMINATION OF OVER-LEARNING AND OVER-FITTING PROBLEM IN BACK PROPAGATION NEURAL NETWORK

A drawback of the error-back propagation algorithm for a multilayer feed forward neural network is over learning or over fitting. We have discussed this problem, and obtained necessary and sufficient Experiment and conditions for over-learning problem to arise. Using those conditions and the concept of a reproducing, this paper proposes methods for choosing training set which is used to prevent over-learning. For a classifier, besides classification capability, its size is another fundamental aspect. In pursuit of high performance, many classifiers do not take into consideration their sizes and contain numerous both essential and insignificant rules. This, however, may bring adverse situation to classifier, for its efficiency will been put down greatly by redundant rules. Hence, it is necessary to eliminate those unwanted rules. We have discussed various experiments with and without over learning or over fitting problem.

[1]  Guoqiang Peter Zhang,et al.  Neural networks for classification: a survey , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[2]  Sanguthevar Rajasekaran,et al.  Neural networks, fuzzy logic, and genetic algorithms : synthesis and applications , 2003 .

[3]  Zheng Niu,et al.  Evolving multi-spectral neural network classifier using a genetic algorithm , 2002, IEEE International Geoscience and Remote Sensing Symposium.

[4]  Manas Ranjan Patra,et al.  NETWORK INTRUSION DETECTION USING NAÏVE BAYES , 2007 .

[5]  George D. Magoulas,et al.  Learning Rate Adaptation in Stochastic Gradient Descent , 2001 .

[6]  Kevin N. Gurney,et al.  An introduction to neural networks , 2018 .

[7]  E. Fernández,et al.  Finding Optimal Neural Network Architecture Using Genetic Algorithms , 2007 .

[8]  Robert A. Schowengerdt,et al.  A detailed comparison of backpropagation neural network and maximum-likelihood classifiers for urban land use classification , 1995, IEEE Trans. Geosci. Remote. Sens..

[9]  Carlos Gershenson,et al.  Artificial Neural Networks for Beginners , 2003, ArXiv.

[10]  Patrick van der Smagt,et al.  Introduction to neural networks , 1995, The Lancet.

[11]  Michael P. Craven.  A faster learning neural network classifier using selective backpropagation , 1997 .

[12]  Teresa Bernarda Ludermir,et al.  An Optimization Methodology for Neural Network Weights and Architectures , 2006, IEEE Transactions on Neural Networks.

[13]  Wenjian Wang,et al.  Optimal feed-forward neural networks based on the combination of constructing and pruning by genetic algorithms , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[14]  Fabrice Rossi,et al.  Recent Advances in the Use of SVM for Functional Data Classification , 2008 .

[15]  Wen Jin,et al.  The improvements of BP neural network learning algorithm , 2000, WCC 2000 - ICSP 2000. 2000 5th International Conference on Signal Processing Proceedings. 16th World Computer Congress 2000.

[16]  S. Selvakani.  Escalate Intrusion Detection using GA - NN , 2009 .