GA-based approaches for finding the minimum reference set for nearest neighbor classification

In this paper, we examine the ability of genetic algorithms to find a compact reference set for nearest neighbor classification. The task of genetic algorithms is to select a small number of reference patterns from a large number of given training patterns. Our pattern selection problem has two objectives: to maximize the classification performance of the reference set and to minimize the size of the reference set. In our genetic algorithm, they are combined into a single scalar fitness function using a constant weight for each objective. Thus our pattern selection problem is handled as a single-objective combinatorial optimization problem with 0-1 variables where "1" means the inclusion of the corresponding pattern in the reference set and "0" means the exclusion. In this paper, we first briefly explain our genetic algorithm for the pattern selection problem for nearest neighbor classification. Next we examine the ability of the genetic algorithm to find a compact reference set by computer simulations on commonly used real-world pattern classification problems. Finally, we suggest some extensions of our genetic algorithm.

[1]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[2]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[3]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[4]  C. G. Hilborn,et al.  The Condensed Nearest Neighbor Rule , 1967 .

[5]  Peter E. Hart,et al.  The condensed nearest neighbor rule (Corresp.) , 1968, IEEE Trans. Inf. Theory.

[6]  Dennis L. Wilson,et al.  Asymptotic Properties of Nearest Neighbor Rules Using Edited Data , 1972, IEEE Trans. Syst. Man Cybern..

[7]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[8]  James M. Keller,et al.  A fuzzy K-nearest neighbor algorithm , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[9]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[10]  Jack Sklansky,et al.  A note on genetic algorithms for large-scale feature selection , 1989, Pattern Recognit. Lett..

[11]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[12]  Lawrence Davis,et al.  Hybridizing the Genetic Algorithm and the K Nearest Neighbors Classification Algorithm , 1991, ICGA.

[13]  Richard J. Enbody,et al.  Further Research on Feature Selection and Classification Using Genetic Algorithms , 1993, ICGA.

[14]  Belur V. Dasarathy,et al.  Minimal consistent set (MCS) identification for optimal nearest neighbor decision systems design , 1994, IEEE Trans. Syst. Man Cybern..

[15]  C. A. Murthy,et al.  Finding a Subset of Representative Points in a Data Set , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[16]  Sandip Sen,et al.  Using real-valued genetic algorithms to evolve rule sets for classification , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[17]  Hideo Tanaka,et al.  Construction of fuzzy classification systems with rectangular fuzzy rules using genetic algorithms , 1994, CVPR 1994.

[18]  Sandip Sen,et al.  PLEASE: A Prototype Learning System Using Genetic Algorithms , 1995, ICGA.

[19]  Ludmila I. Kuncheva,et al.  Editing for the k-nearest neighbors rule by a genetic algorithm , 1995, Pattern Recognit. Lett..

[20]  Hisao Ishibuchi,et al.  Selecting fuzzy if-then rules for classification problems using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[21]  Qiangfu Zhao.  On-line evolutionary learning of NN-MLP based on the attentional learning concept , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[22]  Tatsuo Higuchi,et al.  Minimization of nearest neighbor classifiers based on individual evolutionary algorithm , 1996, Pattern Recognit. Lett..

[23]  L I Kuncheva,et al.  A FUZZY GENERALIZED NEAREST PROTOTYPE CLASSIFIER , 1997 .

[24]  Hisao Ishibuchi,et al.  Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems , 1997, Fuzzy Sets Syst..

[25]  Hisao Ishibuchi,et al.  Evolution of fuzzy nearest neighbor neural networks , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[26]  Ludmila I. Kuncheva,et al.  Fitness functions in editing k-NN reference set by genetic algorithms , 1997, Pattern Recognit..

[27]  Hisao Ishibuchi,et al.  A multi-objective genetic local search algorithm and its application to flowshop scheduling , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[28]  Hisao Ishibuchi,et al.  Improving the performance of fuzzy classifier systems for pattern classification problems with continuous attributes , 1999, IEEE Trans. Ind. Electron..