Feature screening using signal-to-noise ratios

Abstract We present an approach for identifying important input features in multi-layer perceptron feedforward artificial neural networks (ANN) trained via backpropagation. Specifically, we propose a new saliency measure and demonstrate its use in a methodology that holds the promise of potentially identifying and removing noisy input features in a single training run. First, we propose a signal-to-noise ratio (SNR) saliency measure, which determines the saliency of a feature by comparing it to that of an injected noise feature. Next, we propose a feature screening method that utilizes the SNR saliency measure to select a parsimonious set of salient features. The SNR screening method is demonstrated using Fisher's Iris problem. Confidence in the SNR saliency measure and screening method are bolstered by comparisons to Setiono and Liu's neural network feature selector and a principal component analysis (PCA) approach on three real-world applications: the University of Wisconsin Breast Cancer Diagnosis problem, the US Congressional voting records problem, and the Pima Indians Diabetes problem.

[1]  Keinosuke Fukunaga,et al.  Application of the Karhunen-Loève Expansion to Feature Selection and Ordering , 1970, IEEE Trans. Computers.

[2]  Anthony N. Mucciardi,et al.  A Comparison of Seven Techniques for Choosing Subsets of Pattern Recognition Properties , 1971, IEEE Transactions on Computers.

[3]  Donald E. Brown,et al.  Fast generic selection of features for neural network classifiers , 1992, IEEE Trans. Neural Networks.

[4]  Wright-Patterson Afb,et al.  Feature Selection Using a Multilayer Perceptron , 1990 .

[5]  Huan Liu,et al.  Neural-network feature selector , 1997, IEEE Trans. Neural Networks.

[6]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[7]  Gert Pfurtscheller,et al.  Feature selection with distinction sensitive learning vector quantisation and genetic algorithms , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[8]  Kenneth W. Bauer,et al.  Improved feature screening in feedforward neural networks , 1996, Neurocomputing.

[9]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[10]  Todd K. Leen,et al.  Feature selection for improved classification , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[11]  Kenneth W. Bauer,et al.  Integrated feature architecture selection , 1996, IEEE Trans. Neural Networks.

[12]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[13]  Kenneth W. Bauer,et al.  Determining input features for multilayer perceptrons , 1995, Neurocomputing.

[14]  Jack Sklansky,et al.  On Automatic Feature Selection , 1988, Int. J. Pattern Recognit. Artif. Intell..

[15]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[16]  L. Personnaz,et al.  The selection of neural models of nonlinear dynamical systems by statistical tests , 1994, Proceedings of IEEE Workshop on Neural Networks for Signal Processing.

[17]  Tim Watson,et al.  Problems with Using Genetic Algorithms for Neural Network Feature Selection , 1994, ECAI.

[18]  K. Ramachandran,et al.  Mathematical Statistics with Applications. , 1992 .

[19]  T. Adachi,et al.  Feature selection for neural network recognition , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.