Selecting Salient Features for Classification Committees

We present a neural network based approach for identifying salient features for classification in neural network committees. Our approach involves neural network training with an augmented cross-entropy error function. The augmented error function forces the neural network to keep low derivatives of the transfer functions of neurons of the network when learning a classification task. Feature selection is based on two criteria, namely the reaction of the cross-validation data set classification error due to the removal of the individual features and the diversity of neural networks comprising the committee. The algorithm developed removed a large number of features from the original data sets without reducing the classification accuracy of the committees. By contrast, the accuracy of the committees utilizing the reduced feature sets was higher than those exploiting all the original features.