Sparse and low-rank representation for multi-label classification

Multi-label learning deals with the problem where each instance may be associated with multiple labels simultaneously, and how to discover and exploit the label correlations is one of important research issues. In this paper, we propose a novel sparse and low-rank representation-based method for multi-label classification (SLMLC), which can automatically exploit the asymmetric correlations among labels while learning the model parameters in a unified learning framework. More specifically, we assume that the weight matrix is divided into a sparse matrix and a low-rank matrix, where the sparse and low-rank matrices are utilized to capture the specific features that are relevant to each label and the shared feature subspace among all labels, respectively. Then, we integrate multi-label classification and label correlations into a joint learning framework to learn the correlations among labels and the model parameters simultaneously. Lastly, the formulation is transformed into its convex surrogate due to its non-convexity, and we solve it by developing an alternating iterative method. Experimental results on fifteen data sets in terms of six evaluation criteria show that SLMLC achieves superior performance compared to the state-of-the-art multi-label classification algorithms.

[1]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[2]  Lei Wu,et al.  Lift: Multi-Label Learning with Label-Specific Features , 2015, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Eyke Hüllermeier,et al.  Multilabel classification via calibrated label ranking , 2008, Machine Learning.

[4]  Grigorios Tsoumakas,et al.  Multi-Label Classification of Music into Emotions , 2008, ISMIR.

[5]  Jesse Read,et al.  A Pruned Problem Transformation Method for Multi-label Classification , 2008 .

[6]  Jieping Ye,et al.  Learning incoherent sparse and low-rank patterns from multiple tasks , 2010 .

[7]  Grigorios Tsoumakas,et al.  Effective and Efficient Multilabel Classification in Domains with Large Number of Labels , 2008 .

[8]  Sunita Sarawagi,et al.  Discriminative Methods for Multi-labeled Classification , 2004, PAKDD.

[9]  Zhi-Fen He,et al.  Multi-task Joint Feature Selection for Multi-label Classification , 2015 .

[10]  Jieping Ye,et al.  A shared-subspace learning framework for multi-label classification , 2010, TKDD.

[11]  James T. Kwok,et al.  Multilabel Classification with Label Correlations and Missing Labels , 2014, AAAI.

[12]  Gang Chen,et al.  Semi-supervised Multi-label Learning by Solving a Sylvester Equation , 2008, SDM.

[13]  Yuhong Guo,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence Multi-Label Classification Using Conditional Dependency Networks , 2022 .

[14]  Shunxiang Wu,et al.  Multi-label learning based on label-specific features and local pairwise label correlation , 2018, Neurocomputing.

[15]  Stephen P. Boyd,et al.  A rank minimization heuristic with application to minimum order system approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[16]  Wei Xue,et al.  Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence Probabilistic Multi-Label Classification with Sparse Feature Learning , 2022 .

[17]  Zhi-Hua Zhou,et al.  Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization , 2006, IEEE Transactions on Knowledge and Data Engineering.

[18]  Wei Xu,et al.  CNN-RNN: A Unified Framework for Multi-label Image Classification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Nitin J. Janwe,et al.  Multi-label semantic concept detection in videos using fusion of asymmetrically trained deep convolutional neural networks and foreground driven concept co-occurrence matrix , 2017, Applied Intelligence.

[20]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[21]  Geoff Holmes,et al.  Classifier chains for multi-label classification , 2009, Machine Learning.

[22]  Jieping Ye,et al.  An accelerated gradient method for trace norm minimization , 2009, ICML '09.

[23]  Kun Zhang,et al.  Multi-label learning by exploiting label dependency , 2010, KDD.

[24]  Lei Zhang,et al.  Multi-label sparse coding for automatic image annotation , 2009, CVPR.

[25]  Xia Chen,et al.  Multi-Label Classification Based on Low Rank Representation for Image Annotation , 2017, Remote. Sens..

[26]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[27]  Jason Weston,et al.  A kernel method for multi-labelled classification , 2001, NIPS.

[28]  Andrew McCallum,et al.  Collective multi-label classification , 2005, CIKM '05.

[29]  Zhen Wang,et al.  Learning Low-Rank Label Correlations for Multi-label Classification with Missing Labels , 2014, 2014 IEEE International Conference on Data Mining.

[30]  Tao Mei,et al.  Correlative multi-label video annotation , 2007, ACM Multimedia.

[31]  Eyke Hüllermeier,et al.  Label ranking by learning pairwise preferences , 2008, Artif. Intell..

[32]  Ying Wang,et al.  LSTM$$^{2}$$2: Multi-Label Ranking for Document Classification , 2017, Neural Processing Letters.

[33]  Qi Cheng,et al.  Joint multitask feature learning and classifier design , 2013, 2013 47th Annual Conference on Information Sciences and Systems (CISS).

[34]  Yang Yu,et al.  Multi-label hypothesis reuse , 2012, KDD.

[35]  Dit-Yan Yeung,et al.  Multilabel relationship learning , 2013, TKDD.

[36]  Zhi-Hua Zhou,et al.  Multi-Label Learning with Global and Local Label Correlation , 2017, IEEE Transactions on Knowledge and Data Engineering.

[37]  Víctor Robles,et al.  Feature selection for multi-label naive Bayes classification , 2009, Inf. Sci..

[38]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[39]  Jiawei Han,et al.  Correlated multi-label feature selection , 2011, CIKM '11.

[40]  Yoram Singer,et al.  BoosTexter: A Boosting-based System for Text Categorization , 2000, Machine Learning.

[41]  Grigorios Tsoumakas,et al.  Random K-labelsets for Multilabel Classification , 2022 .