Feature Selection for Automatic Classification of Non-Gaussian Data

A computer-based technique for automatic selection of features for the classification of non-Gaussian data is presented. The selection technique exploits interactive cluster finding and a modified branch and bound optimization of piecewise linear classifiers. The technique first finds an efficient set of pairs of oppositely classified clusters to represent the data. Then a zero-one implicit enumeration implements a branch and bound search for a good subset of features. A test of the feature selection technique on multidimensional synthetic and real data yielded close-to-optimum, and in many cases optimum, subsets of features. The real data consisted of a) 1284 12-dimensional feature vectors representing normal and abnormal breast tissue, extracted from X-ray mammograms, and b) 1060 30-dimensional feature vectors representing tanks and clutter in infrared video images.

[1]  Jack Sklansky,et al.  Pattern Classifiers and Trainable Machines , 1981 .

[2]  William S. Meisel,et al.  Computer-oriented approaches to pattern recognition , 1972 .

[3]  Julius T. Tou,et al.  Pattern Recognition Principles , 1974 .

[4]  B. Chandrasekaran,et al.  On dimensionality and sample size in statistical pattern classification , 1971, Pattern Recognit..

[5]  King-Sun Fu,et al.  A Nonparametric Partitioning Procedure for Pattern Classification , 1969, IEEE Transactions on Computers.

[6]  Donald H. Foley Considerations of sample and feature size , 1972, IEEE Trans. Inf. Theory.

[7]  Jerome H. Friedman,et al.  A Recursive Partitioning Decision Rule for Nonparametric Classification , 1977, IEEE Transactions on Computers.

[8]  Jack Sklansky,et al.  The Detection and Segmentation of Blobs in Infrared Images , 1981, IEEE Transactions on Systems, Man, and Cybernetics.

[9]  Olvi L. Mangasarian,et al.  Multisurface method of pattern separation , 1968, IEEE Trans. Inf. Theory.

[10]  Thomas M. Cover,et al.  The Best Two Independent Measurements Are Not the Two Best , 1974, IEEE Trans. Syst. Man Cybern..

[11]  H. Stone Discrete Mathematical Structures and Their Applications , 1973 .

[12]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[13]  Thomas Marill,et al.  On the effectiveness of receptors in recognition systems , 1963, IEEE Trans. Inf. Theory.

[14]  Jack Sklansky,et al.  Training a One-Dimensional Classifier to Minimize the Probability of Error , 1972, IEEE Trans. Syst. Man Cybern..

[15]  A. M. Geoffrion Integer Programming by Implicit Enumeration and Balas’ Method , 1967 .

[16]  M. R. Mickey,et al.  Estimation of Error Rates in Discriminant Analysis , 1968 .

[17]  MANABU ICHINO,et al.  Optimum feature selection by zero-one integer programming , 1984, IEEE Transactions on Systems, Man, and Cybernetics.

[18]  Michael R. Anderberg,et al.  Cluster Analysis for Applications , 1973 .

[19]  W. A. Perkins,et al.  Area Segmentation of Images Using Edge Points , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Keinosuke Fukunaga,et al.  A Branch and Bound Algorithm for Feature Subset Selection , 1977, IEEE Transactions on Computers.

[21]  Laveen N. Kanal,et al.  Patterns in pattern recognition: 1968-1974 , 1974, IEEE Trans. Inf. Theory.

[22]  Jack Sklansky,et al.  Locally Trained Piecewise Linear Classifiers , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  M. Hills Allocation Rules and Their Error Rates , 1966 .

[24]  A. Wayne Whitney,et al.  A Direct Method of Nonparametric Measurement Selection , 1971, IEEE Transactions on Computers.

[25]  E. Balas An Additive Algorithm for Solving Linear Programs with Zero-One Variables , 1965 .

[26]  JAMES C. STOFFEL,et al.  A Classifier Design Technique for Discrete Variable Pattern Recognition Problems , 1974, IEEE Transactions on Computers.

[27]  J. Kittler,et al.  Feature Set Search Alborithms , 1978 .

[28]  King-Sun Fu,et al.  Recent Developments in Pattern Recognition , 1980, IEEE Trans. Computers.

[29]  E. L. Lawler,et al.  Branch-and-Bound Methods: A Survey , 1966, Oper. Res..

[30]  Godfried T. Toussaint,et al.  Bibliography on estimation of misclassification , 1974, IEEE Trans. Inf. Theory.