Pruning methods to MLP neural networks considering proportional apparent error rate for classification problems with unbalanced data

This article deals with classification problems involving unequal probabilities in each class and discusses metrics to systems that use multilayer perceptrons neural networks (MLP) for the task of classifying new patterns. In addition we propose three new pruning methods that were compared to other seven existing methods in the literature for MLP networks. All pruning algorithms presented in this paper have been modified by the authors to do pruning of neurons, in order to produce fully connected MLP networks but being small in its intermediary layer. Experiments were carried out involving the E. coli unbalanced classification problem and ten pruning methods. The proposed methods had obtained good results, actually, better results than another pruning methods previously defined at the MLP neural network area.

[1]  Anil K. Jain,et al.  Parsimonious network design and feature selection through node pruning , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[2]  Giovanna Castellano,et al.  An iterative pruning algorithm for feedforward neural networks , 1997, IEEE Trans. Neural Networks.

[3]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .

[4]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[5]  K. Murase,et al.  A backpropagation algorithm which automatically determines the number of association units , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[6]  Lee Luan Ling,et al.  Optimization of neural classifiers based on Bayesian decision boundaries and idle neurons pruning , 2002, Object recognition supported by user interaction for service robots.

[7]  Yang Wang,et al.  Cost-sensitive boosting for classification of imbalanced data , 2007, Pattern Recognit..

[8]  Ehud D. Karnin,et al.  A simple procedure for pruning back-propagation trained neural networks , 1990, IEEE Trans. Neural Networks.

[9]  Miriam Rodrigues Silvestre,et al.  Statistical Evaluation of Pruning Methods Applied in Hidden Neurons of the MLP Neural Network , 2006, IEEE Latin America Transactions.

[10]  J. Wolfowitz,et al.  Introduction to the Theory of Statistics. , 1951 .

[11]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.