Multi-label core vector machine with a zero label

Abstract Multi-label core vector machine (Rank-CVM) is an efficient and effective algorithm for multi-label classification. But there still exist two aspects to be improved: reducing training and testing computational costs further, and detecting relevant labels effectively. In this paper, we extend Rank-CVM via adding a zero label to construct its variant with a zero label, i.e., Rank-CVMz, which is formulated as the same quadratic programming form with a unit simplex constraint and non-negative ones as Rank-CVM, and then is solved by Frank–Wolfe method efficiently. Attractively, our Rank-CVMz has fewer variables to be solved than Rank-CVM, which speeds up training procedure dramatically. Further, the relevant labels are effectively detected by the zero label. Experimental results on 12 benchmark data sets demonstrate that our method achieves a competitive performance, compared with six existing multi-label algorithms according to six indicative instance-based measures. Moreover, on the average, our Rank-CVMz runs 83 times faster and has slightly fewer support vectors than its origin Rank-CVM.

[1]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[2]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[3]  Eyke Hüllermeier,et al.  Multilabel classification via calibrated label ranking , 2008, Machine Learning.

[4]  Grigorios Tsoumakas,et al.  Multi-Label Classification , 2009, Database Technologies: Concepts, Methodologies, Tools, and Applications.

[5]  Eyke Hüllermeier,et al.  Combining Instance-Based Learning and Logistic Regression for Multilabel Classification , 2009, ECML/PKDD.

[6]  David G. Stork,et al.  Pattern Classification , 1973 .

[7]  Eyke Hüllermeier,et al.  On label dependence and loss minimization in multi-label classification , 2012, Machine Learning.

[8]  José Ramón Quevedo,et al.  Multilabel classifiers with a probabilistic thresholding strategy , 2012, Pattern Recognit..

[9]  Ivor W. Tsang,et al.  Core Vector Machines: Fast SVM Training on Very Large Data Sets , 2005, J. Mach. Learn. Res..

[10]  Zhi-Hua Zhou,et al.  Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization , 2006, IEEE Transactions on Knowledge and Data Engineering.

[11]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[12]  Jianhua Xu,et al.  An efficient multi-label support vector machine with a zero label , 2012, Expert Syst. Appl..

[13]  Saso Dzeroski,et al.  An extensive experimental comparison of methods for multi-label learning , 2012, Pattern Recognit..

[14]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[15]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[16]  Fernando Benites,et al.  Multi-label classification and extracting predicted class hierarchies , 2011, Pattern Recognit..

[17]  Patrice Marcotte,et al.  Some comments on Wolfe's ‘away step’ , 1986, Math. Program..

[18]  Jianhua Xu,et al.  An extended one-versus-rest support vector machine for multi-label classification , 2011, Neurocomputing.

[19]  Grigorios Tsoumakas,et al.  Multi-label classification of music by emotion , 2011 .

[20]  Chih-Jen Lin,et al.  Training v-Support Vector Classifiers: Theory and Algorithms , 2001, Neural Computation.

[21]  Geoff Holmes,et al.  Classifier chains for multi-label classification , 2009, Machine Learning.

[22]  Alex Alves Freitas,et al.  A Tutorial on Multi-label Classification Techniques , 2009, Foundations of Computational Intelligence.

[23]  Xian-Sheng Hua,et al.  A transductive multi-label learning approach for video concept detection , 2011, Pattern Recognit..

[24]  Jianhua Xu,et al.  A Random Block Coordinate Descent Method for Multi-label Support Vector Machine , 2013, ICONIP.

[25]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[26]  Lior Rokach,et al.  Data Mining and Knowledge Discovery Handbook, 2nd ed , 2010, Data Mining and Knowledge Discovery Handbook, 2nd ed..

[27]  Yuanping Zhu,et al.  Calibrated Rank-SVM for multi-label image categorization , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[28]  Jianhua Xu,et al.  Fast multi-label core vector machine , 2013, Pattern Recognit..

[29]  Fabio Roli,et al.  Threshold optimisation for multi-label classifiers , 2013, Pattern Recognit..

[30]  Jianhua Xu,et al.  Random block coordinate descent method for multi-label support vector machine with a zero label , 2014, Expert Syst. Appl..

[31]  Grigorios Tsoumakas,et al.  Mining Multi-label Data , 2010, Data Mining and Knowledge Discovery Handbook.

[32]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[33]  Jason Weston,et al.  A kernel method for multi-labelled classification , 2001, NIPS.

[34]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[35]  Philip Wolfe,et al.  An algorithm for quadratic programming , 1956 .

[36]  Yoram Singer,et al.  BoosTexter: A Boosting-based System for Text Categorization , 2000, Machine Learning.

[37]  Grigorios Tsoumakas,et al.  Random K-labelsets for Multilabel Classification , 2022 .