Improving performance of nearest neighborhood classifier using genetic programming

Nearest neighborhood classifier (kNN) is most widely used in pattern recognition applications. Depending on the selection of voting methodology, the problem of outliers has been encountered in this classifier. Therefore, selection and optimization of the voting methodology is very important. In this work, we have used Genetic Programming (GP) to improve the performance of nearest neighbor classifier. Instead of using predefined k nearest neighbors, the number of men and women in the first two quartiles in Euclidean space are used for voting. GP is, then, used to evolve an optimal class mapping function that effectively reduces the outliers. The performance of modified nearest neighborhood (ModNN) classifier is then compared with the conventional kNN for gender classification problem. Receiver Operating Characteristics curve and its Area Under the Convex Hull (A UCH) are used as the performance measures. Considering the first three and first five eigen features respectively, ModNN achieves AUCH equal to 0.985 and 0.992 as compared to 0.9693 and 0.9795 of conventional kNN respectively.

[1]  A. M. Burton,et al.  What's the Difference between Men and Women? Evidence from Facial Measurement , 1993, Perception.

[2]  William B. Langdon,et al.  Genetic programming for combining classifiers , 2001 .

[3]  Dimitrios Gunopulos,et al.  Locally Adaptive Metric Nearest-Neighbor Classification , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Venu Govindaraju,et al.  Improvements in K-Nearest Neighbor Classification , 2001, ICAPR.

[5]  Lawrence Sirovich,et al.  Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  David G. Stork,et al.  Pattern Classification , 1973 .

[7]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[8]  George Bebis,et al.  Neural-network-based gender classification using genetic search for eigen-feature selection , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[9]  Peter Nordin,et al.  Genetic programming - An Introduction: On the Automatic Evolution of Computer Programs and Its Applications , 1998 .

[10]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[11]  A.M. Mirza,et al.  Gender classification using discrete cosine transformation: a comparison of different classifiers , 2003, 7th International Multi Topic Conference, 2003. INMIC 2003..

[12]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[13]  Michael C. Lincoln.  Pose-independent face recognition , 2003 .

[14]  A J O'Toole,et al.  More about the Difference between Men and Women: Evidence from Linear Neural Networks and the Principal-Component Approach , 1995, Perception.

[15]  Keinosuke Fukunaga,et al.  The optimal distance measure for nearest neighbor classification , 1981, IEEE Trans. Inf. Theory.

[16]  Thomas G. Dietterich,et al.  Locally Adaptive Nearest Neighbor Algorithms , 1993, NIPS.

[17]  Asifullah Khan,et al.  Combination and optimization of classifiers in gender classification using genetic programming , 2005 .

[18]  Mahesan Niranjan,et al.  Realisable Classifiers: Improving Operating Performance on Variable Cost Problems , 1998, BMVC.

[19]  Ming-Hsuan Yang,et al.  Learning Gender with Support Faces , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  S. Salzberg,et al.  A weighted nearest neighbor algorithm for learning with symbolic features , 2004, Machine Learning.

[21]  Robert Tibshirani,et al.  Discriminant Adaptive Nearest Neighbor Classification and Regression , 1995, NIPS.