Partial multi-label learning with noisy side information

Partial multi-label learning (PML) aims to learn from the training data where each training example is annotated with a candidate label set, among which only a subset is relevant. Despite the success of existing PML approaches, a major drawback of them lies in lacking of robustness to noisy side information. To tackle this problem, we introduce a novel partial multi-label learning with noisy side information approach, which simultaneously removes noisy outliers from the training instances and trains robust partial multi-label classifier for unlabeled instances prediction. Specifically, we first represent the observed sample set as a feature matrix and then decompose it into an ideal feature matrix and an outlier feature matrix by using the low-rank and sparse decomposition scheme, where the former is constrained to be low rank by considering that the noise-free feature information always lies in a low-dimensional subspace and the latter is assumed to be sparse by considering that the outliers are usually sparse among the observed feature matrix. Secondly, we refine an ideal label confidence matrix from the observed label matrix and use the graph Laplacian regularization to constrain the confidence matrix to keep the intrinsic structure among feature vectors. Thirdly, we constrain the feature mapping matrix to be low rank by utilizing the label correlations. Finally, we obtain both the ideal features and ground-truth labels via minimizing the loss function, where the augmented Lagrange multiplier algorithm and quadratic programming are incorporated to solve the optimization problem. Extensive experiments conducted on ten different data sets demonstrate the effectiveness of our proposed method.

[1]  Min-Ling Zhang,et al.  Lift: Multi-Label Learning with Label-Specific Features. , 2015, IEEE transactions on pattern analysis and machine intelligence.

[2]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[3]  Congyan Lang,et al.  Graph regularized low-rank feature mapping for multi-label learning with application to image annotation , 2018, Multidimens. Syst. Signal Process..

[4]  Johannes Fürnkranz,et al.  Efficient Pairwise Multilabel Classification for Large-Scale Problems in the Legal Domain , 2008, ECML/PKDD.

[5]  Gert R. G. Lanckriet,et al.  Semantic Annotation and Retrieval of Music and Sound Effects , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[6]  Alan L. Yuille,et al.  The Concave-Convex Procedure , 2003, Neural Computation.

[7]  Jun Wang,et al.  Feature-Induced Partial Multi-label Learning , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[8]  K. Bretonnel Cohen,et al.  A shared task involving multi-label classification of clinical free text , 2007, BioNLP@ACL.

[9]  Min-Ling Zhang,et al.  Partial Multi-Label Learning via Credible Label Elicitation , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Weiwei Liu,et al.  Discriminative and Correlative Partial Multi-Label Learning , 2019, IJCAI.

[11]  Tao Wang,et al.  Partial Multi-Label Learning by Low-Rank and Sparse Decomposition , 2019, AAAI.

[12]  Grigorios Tsoumakas,et al.  Protein Classification with Multiple Algorithms , 2005, Panhellenic Conference on Informatics.

[13]  Alexandre Bernardino,et al.  Matrix Completion for Weakly-Supervised Multi-Label Image Classification , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Yidong Li,et al.  Partial Multi-Label Learning via Probabilistic Graph Matching Mechanism , 2020, KDD.

[15]  Jieping Ye,et al.  An accelerated gradient method for trace norm minimization , 2009, ICML '09.

[16]  Nenghai Yu,et al.  Non-negative low rank and sparse graph for semi-supervised learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Lei Wu,et al.  Lift: Multi-Label Learning with Label-Specific Features , 2015, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Inderjit S. Dhillon,et al.  Matrix Completion with Noisy Side Information , 2015, NIPS.

[19]  Miao Xu,et al.  Speedup Matrix Completion with Side Information: Application to Multi-Label Learning , 2013, NIPS.

[20]  Zhixun Su,et al.  Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation , 2011, NIPS.

[21]  Xiangliang Zhang,et al.  Multi-label Learning with Highly Incomplete Data via Collaborative Embedding , 2018, KDD.

[22]  Fernando De la Torre,et al.  Robust Regression , 2016, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Daming Shi,et al.  Low-Rank-Sparse Subspace Representation for Robust Regression , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Yidong Li,et al.  Noisy label tolerance: A new perspective of Partial Multi-Label Learning , 2021, Inf. Sci..

[25]  Bo An,et al.  Partial Label Learning by Semantic Difference Maximization , 2019, IJCAI.

[26]  David A. Forsyth,et al.  Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary , 2002, ECCV.

[27]  Ziwei Li,et al.  Partial Multi-Label Learning via Multi-Subspace Representation , 2020, IJCAI.

[28]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[29]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[30]  Ning Xu,et al.  Partial Multi-Label Learning with Label Distribution , 2020, AAAI.

[31]  Yao Hu,et al.  Feature-Induced Manifold Disambiguation for Multi-View Partial Multi-label Learning , 2020, KDD.

[32]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[33]  Bo An,et al.  Leveraging Latent Label Distributions for Partial Label Learning , 2018, IJCAI.