Evidential Editing K-Nearest Neighbor Classifier

One of the difficulties that arises when using the K-nearest neighbor rule is that each of the labeled training samples is given equal importance in deciding the class of the query pattern to be classified, regardless of their typicality. In this paper, the theory of belief functions is introduced into the K-nearest neighbor rule to develop an evidential editing version of this algorithm. An evidential editing procedure is proposed to reassign the original training samples with new labels represented by an evidential membership structure. With the introduction of the evidential editing procedure, the uncertainty of noisy patterns or samples in overlapping regions can be well characterized. After the evidential editing, a classification procedure is developed to handle the more general situation in which the edited training samples are assigned dependent evidential labels. Two experiments based on synthetic and real data sets were carried out to show the effectiveness of the proposed method.

[1]  Jack Koplowitz,et al.  On the relation of performance to editing in nearest neighbor rules , 1981, Pattern Recognit..

[2]  Arthur P. Dempster,et al.  Upper and Lower Probabilities Induced by a Multivalued Mapping , 1967, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[3]  James M. Keller,et al.  A fuzzy K-nearest neighbor algorithm , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[4]  Thierry Denoeux,et al.  Editing training data for multi-label classification with the k-nearest neighbor rule , 2016, Pattern Analysis and Applications.

[5]  Dennis L. Wilson,et al.  Asymptotic Properties of Nearest Neighbor Rules Using Edited Data , 1972, IEEE Trans. Syst. Man Cybern..

[6]  Quan Pan,et al.  A new belief-based K-nearest neighbor classification method , 2013, Pattern Recognit..

[7]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[8]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[9]  Quan Pan,et al.  Fusion of pairwise nearest-neighbor classifiers based on pairwise-weighted distance metric and Dempster-Shafer theory , 2014, 17th International Conference on Information Fusion (FUSION).

[10]  Thierry Denoeux.  A k -Nearest Neighbor Classification Rule Based on Dempster-Shafer Theory , 2008, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[11]  Robert LIN,et al.  NOTE ON FUZZY SETS , 2014 .

[12]  Philippe Smets,et al.  Decision making in the TBM: the necessity of the pignistic transformation , 2005, Int. J. Approx. Reason..

[13]  J. L. Hodges,et al.  Discriminatory Analysis - Nonparametric Discrimination: Consistency Properties , 1989 .

[14]  Thierry Denoeux,et al.  Representing uncertainty on set-valued variables using belief functions , 2010, Artif. Intell..

[15]  T. Denœux.  Conjunctive and disjunctive combination of belief functions induced by nondistinct bodies of evidence , 2008 .

[16]  Larry D. Hostetler,et al.  k-nearest-neighbor Bayes-risk estimation , 1975, IEEE Trans. Inf. Theory.

[17]  Philippe Smets,et al.  Classification Using Belief Functions: Relationship Between Case-Based and Model-Based Approaches , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).