Improved pseudo nearest neighbor classification

k-Nearest neighbor (KNN) rule is a very simple and powerful classification algorithm. In this article, we propose a new KNN-based classifier, called the local mean-based pseudo nearest neighbor (LMPNN) rule. It is motivated by the local mean-based k-nearest neighbor (LMKNN) rule and the pseudo nearest neighbor (PNN) rule, with the aim of improving the classification performance. In the proposed LMPNN, the k nearest neighbors from each class are searched as the class prototypes, and then the local mean vectors of the neighbors are yielded. Subsequently, we attempt to find the local mean-based pseudo nearest neighbor per class by employing the categorical k local mean vectors, and classify the unknown query patten according to the distances between the query and the pseudo nearest neighbors. To assess the classification performance of the proposed LMPNN, it is compared with the competing classifiers, such as LMKNN and PNN, in terms of the classification error on thirty-two real UCI data sets, four artificial data sets and three image data sets. The comprehensively experimental results suggest that the proposed LMPNN classifier is a promising algorithm in pattern recognition.

[1]  Fernando Fernández,et al.  Local Feature Weighting in Nearest Prototype Classification , 2008, IEEE Transactions on Neural Networks.

[2]  Hongwei Zhu,et al.  An adaptive fuzzy evidential nearest neighbor formulation for classifying remote sensing images , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Jie Xu,et al.  A nonnegative sparse representation based fuzzy similar neighbor classifier , 2013, Neurocomputing.

[4]  Zheng Bao,et al.  Large margin nearest local mean classifier , 2010, Signal Process..

[5]  Zhi-gang Su,et al.  Improved adaptive evidential k-NN rule and its application for monitoring level of coal powder filling in ball mill , 2009 .

[6]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[7]  Reza Ebrahimpour,et al.  Combining complementary information sources in the Dempster-Shafer framework for solving classification problems with imperfect labels , 2012, Knowl. Based Syst..

[8]  Sahibsingh A. Dudani.  The Distance-Weighted k-Nearest-Neighbor Rule , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[9]  M. Narasimha Murty,et al.  Fusion of multiple approximate nearest neighbor classifiers for fast and efficient classification , 2004, Inf. Fusion.

[10]  Yoshihiko Hamamoto,et al.  A local mean-based nonparametric classifier , 2006, Pattern Recognit. Lett..

[11]  Thierry Denoeux,et al.  An evidence-theoretic k-NN rule with parameter optimization , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[12]  Dimitrios Gunopulos,et al.  Locally Adaptive Metric Nearest-Neighbor Classification , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  S. García,et al.  An Extension on "Statistical Comparisons of Classifiers over Multiple Data Sets" for all Pairwise Comparisons , 2008 .

[14]  Yan Qiu Chen,et al.  The Nearest Neighbor Algorithm of Local Probability Centers , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  James M. Keller,et al.  A fuzzy K-nearest neighbor algorithm , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[16]  Leon N. Cooper,et al.  Neighborhood size selection in the k-nearest-neighbor rule using statistical confidence , 2006, Pattern Recognit..

[17]  Thierry Denoeux,et al.  A k-nearest neighbor classification rule based on Dempster-Shafer theory , 1995, IEEE Trans. Syst. Man Cybern..

[18]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[19]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[20]  Jianping Gou,et al.  A Novel Weighted Voting for K-Nearest Neighbor Rule , 2011, J. Comput..

[21]  Francisco Herrera,et al.  A Taxonomy and Experimental Study on Prototype Generation for Nearest Neighbor Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[22]  Horst Bischof,et al.  Optimizing 1-Nearest Prototype Classifiers , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Haisheng Li,et al.  Random subspace evidence classifier , 2013, Neurocomputing.

[24]  Hong Liu,et al.  Coarse to fine K nearest neighbor classifier , 2013, Pattern Recognit. Lett..

[25]  José Salvador Sánchez,et al.  High training set size reduction by space partitioning and prototype abstraction , 2004, Pattern Recognit..

[26]  David Zhang,et al.  Bimodal biometrics based on a representation and recognition approach , 2011 .

[27]  Jianping Gou,et al.  A Local Mean-Based k-Nearest Centroid Neighbor Classifier , 2012, Comput. J..

[28]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[29]  Klaus Obermayer,et al.  Soft nearest prototype classification , 2003, IEEE Trans. Neural Networks.

[30]  David Zhang,et al.  On kernel difference-weighted k-nearest neighbor classification , 2008, Pattern Analysis and Applications.

[31]  Francisco Herrera,et al.  Prototype Selection for Nearest Neighbor Classification: Taxonomy and Empirical Study , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Mark L. Nagurka,et al.  A vector quantization method for nearest neighbor classifier design , 2004, Pattern Recognit. Lett..

[33]  Jian Yang,et al.  Linear reconstruction measure steered nearest neighbor classification framework , 2014, Pattern Recognit..

[34]  Stan Sclaroff,et al.  Efficient nearest neighbor classification using a cascade of approximate similarity measures , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[35]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[36]  Terry J. Wagner,et al.  Convergence of the nearest neighbor rule , 1970, IEEE Trans. Inf. Theory.

[37]  Jian Yang,et al.  From classifiers to discriminators: A nearest neighbor rule induced discriminant analysis , 2011, Pattern Recognit..

[38]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[39]  Robert P. W. Duin,et al.  Prototype selection for dissimilarity-based classifiers , 2006, Pattern Recognit..

[40]  Georgios Evangelidis,et al.  Fast and Accurate k-Nearest Neighbor Classification Using Prototype Selection by Clustering , 2012, 2012 16th Panhellenic Conference on Informatics.

[41]  Francisco Herrera,et al.  Fuzzy nearest neighbor algorithms: Taxonomy, experimental analysis and prospects , 2014, Inf. Sci..

[42]  Yong Zeng,et al.  Nonparametric classification based on local mean and class statistics , 2009, Expert Syst. Appl..

[43]  Yong Zeng,et al.  Pseudo nearest neighbor rule for pattern classification , 2009, Expert Syst. Appl..

[44]  Reza Ebrahimpour,et al.  Combining classifiers using nearest decision prototypes , 2013, Appl. Soft Comput..

[45]  Jianping Gou,et al.  A new distance-weighted k-nearest neighbor classifier , 2012 .

[46]  Robert Tibshirani,et al.  Discriminant Adaptive Nearest Neighbor Classification , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.