Cost-sensitive multi-label learning with positive and negative label pairwise correlations

Multi-label learning is the problem where each instance is associated with multiple labels simultaneously. Binary Relevance (BR) is a representative algorithm for multi-label learning. However, it may suffer the class-imbalance issue especially when the label space is large. Besides, it ignores the label correlations, which is of importance to improve the performance. Moreover, labels might have positive and negative correlations in real applications, but existing methods seldom exploit the negative label correlations. In this paper, we propose a novel Cost-sensitive multi-label learning model with Positive and Negative Label pairwise correlations (CPNL), which extends BR to tackle the above issues. The kernel extension of the linear model is also provided to explore complex input-output relationships. Moreover, we adopt two accelerated gradient methods (AGM) to efficiently solve the linear and kernel models. Experimental results show that our approach CPNL achieves a competitive performance to some state-of-the-art approaches for multi-label learning.

[1]  Zhi-Hua Zhou,et al.  Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization , 2006, IEEE Transactions on Knowledge and Data Engineering.

[2]  Guoqiang Wu,et al.  A unified framework implementing linear binary relevance for multi-label learning , 2018, Neurocomputing.

[3]  Eyke Hüllermeier,et al.  Dependent binary relevance models for multi-label classification , 2014, Pattern Recognit..

[4]  Wei Xu,et al.  CNN-RNN: A Unified Framework for Multi-label Image Classification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[6]  Liang Lin,et al.  Recurrent Attentional Reinforcement Learning for Multi-label Image Recognition , 2017, AAAI.

[7]  Min-Ling Zhang,et al.  Feature-Induced Labeling Information Enrichment for Multi-Label Learning , 2018, AAAI.

[8]  Geoff Holmes,et al.  Classifier chains for multi-label classification , 2009, Machine Learning.

[9]  Jieping Ye,et al.  An accelerated gradient method for trace norm minimization , 2009, ICML '09.

[10]  Tao Mei,et al.  Correlative multi-label video annotation , 2007, ACM Multimedia.

[11]  Chris H. Q. Ding,et al.  Image annotation using multi-label correlated Green's function , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[12]  Zhen Wang,et al.  Learning Low-Rank Label Correlations for Multi-label Classification with Missing Labels , 2014, 2014 IEEE International Conference on Data Mining.

[13]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[14]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[15]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[16]  Dacheng Tao,et al.  Local Rademacher Complexity for Multi-Label Learning , 2014, IEEE Transactions on Image Processing.

[17]  Kun Zhang,et al.  Multi-label learning by exploiting label dependency , 2010, KDD.

[18]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[19]  Xindong Wu,et al.  Learning Label Specific Features for Multi-label Classification , 2015, 2015 IEEE International Conference on Data Mining.

[20]  Naonori Ueda,et al.  Parametric Mixture Models for Multi-Labeled Text , 2002, NIPS.

[21]  Andrew McCallum,et al.  Collective multi-label classification , 2005, CIKM '05.

[22]  Amanda Clare,et al.  Knowledge Discovery in Multi-label Phenotype Data , 2001, PKDD.

[23]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[24]  Heikki Huttunen,et al.  Polyphonic sound event detection using multi label deep neural networks , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[25]  Grigorios Tsoumakas,et al.  Mining Multi-label Data , 2010, Data Mining and Knowledge Discovery Handbook.

[26]  Yoram Singer,et al.  BoosTexter: A Boosting-based System for Text Categorization , 2000, Machine Learning.

[27]  Grigorios Tsoumakas,et al.  Random K-labelsets for Multilabel Classification , 2022 .

[28]  Kim-Chuan Toh,et al.  A Unified Formulation and Fast Accelerated Proximal Gradient Method for Classification , 2017, J. Mach. Learn. Res..

[29]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[30]  Lei Wu,et al.  Lift: Multi-Label Learning with Label-Specific Features , 2015, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[32]  Xin Geng,et al.  Leveraging Implicit Relative Labeling-Importance Information for Effective Multi-label Learning , 2015, 2015 IEEE International Conference on Data Mining.

[33]  Zhi-Hua Zhou,et al.  Multi-Label Learning with Global and Local Label Correlation , 2017, IEEE Transactions on Knowledge and Data Engineering.

[34]  Sunita Sarawagi,et al.  Discriminative Methods for Multi-labeled Classification , 2004, PAKDD.

[35]  Qingming Huang,et al.  Multi-label classification by exploiting local positive and negative pairwise label correlation , 2017, Neurocomputing.

[36]  Xindong Wu,et al.  NESVM: A Fast Gradient Method for Support Vector Machines , 2010, 2010 IEEE International Conference on Data Mining.

[37]  Grigorios Tsoumakas,et al.  MULAN: A Java Library for Multi-Label Learning , 2011, J. Mach. Learn. Res..

[38]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[39]  Zhi-Hua Zhou,et al.  Ensemble Methods: Foundations and Algorithms , 2012 .

[40]  Donghwan Kim,et al.  Optimized first-order methods for smooth convex minimization , 2014, Mathematical Programming.

[41]  Xin Geng,et al.  Binary relevance for multi-label learning: an overview , 2018, Frontiers of Computer Science.

[42]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[43]  Bingbing Ni,et al.  HCP: A Flexible CNN Framework for Multi-Label Image Classification , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[45]  Eyke Hüllermeier,et al.  Multilabel classification via calibrated label ranking , 2008, Machine Learning.

[46]  M. Friedman.  The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .