Feature selection for neural network classifiers using saliency and genetic algorithms

In this paper the authors present the results of a research investigation on feature selection methods for neural network classifiers. As problems presented to computers for analysis become more complex and data dimensionality grows in size, traditional methods of feature extraction are being taxed beyond the limits of their usefulness. New methods of feature selection show promise in the laboratory, but need to be proven with real-world solutions. The purpose of this research is to compare the performance of newly proposed methods of selecting features on three challenging problems using non- artificial data. A feature saliency technique, and several variants of genetic algorithms, and random feature selection are compared and contrasted.

[1]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[2]  E. DeRouin,et al.  Hyperspectral data fusion for target detection using neural networks , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[3]  Paul F. M. J. Verschure,et al.  A note on chaotic behavior in simple neural networks , 1990, Neural Networks.

[4]  Godfried T. Toussaint,et al.  Bibliography on estimation of misclassification , 1974, IEEE Trans. Inf. Theory.

[5]  Donald E. Brown,et al.  Fast generic selection of features for neural network classifiers , 1992, IEEE Trans. Neural Networks.

[6]  Babak Hassibi,et al.  Second Order Derivatives for Network Pruning: Optimal Brain Surgeon , 1992, NIPS.

[7]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[8]  Warren S. Sarle,et al.  Stopped Training and Other Remedies for Overfitting , 1995 .

[9]  Terrence J. Sejnowski,et al.  Analysis of hidden units in a layered network trained to classify sonar targets , 1988, Neural Networks.

[10]  M. Eslami,et al.  Introduction to System Sensitivity Theory , 1980, IEEE Transactions on Systems, Man, and Cybernetics.

[11]  T. Gal,et al.  Linear parametric programming—A brief survey , 1984 .

[12]  Gilbert Syswerda,et al.  Uniform Crossover in Genetic Algorithms , 1989, ICGA.

[13]  Thomas Marill,et al.  On the effectiveness of receptors in recognition systems , 1963, IEEE Trans. Inf. Theory.

[14]  W. T. Illingworth,et al.  Practical guide to neural nets , 1991 .

[15]  R. Hecht-Nielsen,et al.  Theory of the Back Propagation Neural Network , 1989 .

[16]  George S. Sebestyen,et al.  Decision-making processes in pattern recognition , 1962 .

[17]  John Holland,et al.  Adaptation in Natural and Artificial Sys-tems: An Introductory Analysis with Applications to Biology , 1975 .

[18]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .