A Ranking-based KNN Approach for Multi-Label Classification

Multi-label classication has attracted a great deal of attention in recent years. This paper presents an interesting nding, namely, being able to identify neighbors with trustable labels can signicantly improve the classication accuracy. Based on this nding, we propose a k-nearest-neighbor-based ranking approach to solve the multi-label classication problem. The approach exploits a ranking model to learn which neighbor’s labels are more trustable candidates for a weighted KNN-based strategy, and then assigns higher weights to those candidates when making weighted-voting decisions. The weights can then be determined by using a generalized pattern search technique. We collect several real-word data sets from various domains for the experiment. Our experiment results demonstrate that the proposed method outperforms state-of-the-art instance-based learning approaches. We believe that appropriately exploiting k-nearest neighbors is useful to solve the multi-label problem.

[1]  Grigorios Tsoumakas,et al.  Multi-Label Classification of Music into Emotions , 2008, ISMIR.

[2]  Grigorios Tsoumakas,et al.  Random K-labelsets for Multilabel Classification , 2022 .

[3]  Charles Audet Convergence Results for Generalized Pattern Search Algorithms are Tight , 2004 .

[4]  Hsin-Min Wang,et al.  Homogeneous segmentation and classifier ensemble for audio tag annotation and retrieval , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[5]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[6]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[7]  Tamara G. Kolda,et al.  Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods , 2003, SIAM Rev..

[8]  Robert Michael Lewis,et al.  Pattern Search Algorithms for Bound Constrained Minimization , 1999, SIAM J. Optim..

[9]  Saso Dzeroski,et al.  An extensive experimental comparison of methods for multi-label learning , 2012, Pattern Recognit..

[10]  Eyke Hüllermeier,et al.  Combining Instance-Based Learning and Logistic Regression for Multilabel Classification , 2009, ECML/PKDD.

[11]  D. Sculley,et al.  Large Scale Learning to Rank , 2009 .

[12]  Sahibsingh A. Dudani The Distance-Weighted k-Nearest-Neighbor Rule , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[13]  Jason Weston,et al.  A kernel method for multi-labelled classification , 2001, NIPS.

[14]  Koby Crammer,et al.  Online Passive-Aggressive Algorithms , 2003, J. Mach. Learn. Res..

[15]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[16]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..