Weighted Nearest Neighbor Classification via Maximizing Classification Consistency

The nearest neighbor classification is a simple and effective technique for pattern recognition. The performance of this technique is known to be sensitive to the distance function used in classifying a test instance. In this paper, we propose a technique to learn sample weights via maximizing classification consistency. Experimental analysis shows that the distance trained in this way enlarges the classification consistency on several datasets and has a strong ability to tolerate noise. Moreover, the proposed approach has better performance than nearest neighbor classification and several state-of-the-art methods.

[1]  Yiyu Yao,et al.  Attribute reduction in decision-theoretic rough set models , 2008, Inf. Sci..

[2]  Leon N. Cooper,et al.  Improving nearest neighbor rule with a simple adaptive distance measure , 2006, Pattern Recognit. Lett..

[3]  Qinghua Hu,et al.  Hybrid attribute reduction based on a novel fuzzy-rough model and information granulation , 2007, Pattern Recognit..

[4]  Ron Kohavi,et al.  The Utility of Feature Weighting in Nearest-Neighbor Algorithms , 1997 .

[5]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[6]  Claire Cardie,et al.  Examining Locally Varying Weights for Nearest Neighbor Algorithms , 1997, ICCBR.

[7]  Dominik Lzak.  Degrees of conditional (in)dependence: A framework for approximate Bayesian networks and examples related to the rough set-based feature selection , 2009 .

[8]  Robert Tibshirani,et al.  Discriminant Adaptive Nearest Neighbor Classification and Regression , 1995, NIPS.

[9]  Gerhard Widmer,et al.  Machine Learning: ECML-97 , 1997, Lecture Notes in Computer Science.

[10]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[11]  Nehad N. Morsi,et al.  Axiomatics for fuzzy rough sets , 1998, Fuzzy Sets Syst..

[12]  Christian A. Rees,et al.  Molecular portraits of human breast tumours , 2000, Nature.

[13]  Dennis L. Wilson,et al.  Asymptotic Properties of Nearest Neighbor Rules Using Edited Data , 1972, IEEE Trans. Syst. Man Cybern..

[14]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[15]  Luc Lamontagne,et al.  Case-Based Reasoning Research and Development , 1997, Lecture Notes in Computer Science.

[16]  Enrique Vidal,et al.  Learning weighted metrics to minimize nearest-neighbor classification error , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Naftali Tishby,et al.  Margin based feature selection - theory and algorithms , 2004, ICML.