An Evidence-Theoretic k-Nearest Neighbor Rule for Multi-label Classification

In multi-label learning, each instance in the training set is associated with a set of labels, and the task is to output a label set for each unseen instance. This paper describes a new method for multi-label classification based on the Dempster-Shafer theory of belief functions to classify an unseen instance on the basis of its k nearest neighbors. The proposed method generalizes an existing single-label evidence-theoretic learning method to the multi-label case. In multi-label case, the frame of discernment is not the set of all possible classes, but it is the powerset of this set. That requires an extension of evidence theory to manipulate multi-labelled data. Using evidence theory makes us able to handle ambiguity and imperfect knowledge regarding the label sets of training patterns. Experiments on benchmark datasets show the efficiency of the proposed approach as compared to other existing methods.

[1]  Koby Crammer,et al.  A Family of Additive Online Algorithms for Category Ranking , 2003, J. Mach. Learn. Res..

[2]  Philippe Smets,et al.  The Combination of Evidence in the Transferable Belief Model , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Yoram Singer,et al.  BoosTexter: A Boosting-based System for Text Categorization , 2000, Machine Learning.

[4]  Volker Roth,et al.  Kernel methods for regression and classification , 2001 .

[5]  Thierry Denoeux,et al.  Handling possibilistic labels in pattern classification using evidential reasoning , 2001, Fuzzy Sets Syst..

[6]  Grigorios Tsoumakas,et al.  Multi-Label Classification of Music into Emotions , 2008, ISMIR.

[7]  Thierry Denoeux A k -Nearest Neighbor Classification Rule Based on Dempster-Shafer Theory , 2008, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[8]  Yiming Yang,et al.  An Evaluation of Statistical Approaches to Text Categorization , 1999, Information Retrieval.

[9]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[10]  Koby Crammer,et al.  A new family of online algorithms for category ranking , 2002, SIGIR '02.

[11]  Thierry Denoeux,et al.  Multi-label classification algorithm derived from K-nearest neighbor rule with label dependencies , 2008, 2008 16th European Signal Processing Conference.

[12]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[13]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[14]  Philippe Smets,et al.  Classification Using Belief Functions: Relationship Between Case-Based and Model-Based Approaches , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..