Lift: Multi-Label Learning with Label-Specific Features

Multi-label learning deals with the problem where each example is represented by a single instance (feature vector) while associated with a set of class labels. Existing approaches learn from multi-label data by manipulating with identical feature set, i.e. the very instance representation of each example is employed in the discrimination processes of all class labels. However, this popular strategy might be suboptimal as each label is supposed to possess specific characteristics of its own. In this paper, another strategy to learn from multi-label data is studied, where label-specific features are exploited to benefit the discrimination of different class labels. Accordingly, an intuitive yet effective algorithm named LIFT, i.e. multi-label learning with Label specific Features, is proposed. LIFT firstly constructs features specific to each label by conducting clustering analysis on its positive and negative instances, and then performs training and testing by querying the clustering results. Comprehensive experiments on a total of 17 benchmark data sets clearly validate the superiority of LIFT against other well-established multi-label learning algorithms as well as the effectiveness of label-specific features.

[1]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[2]  Shou-De Lin,et al.  Cost-Sensitive Multi-Label Learning for Audio Tag Annotation and Retrieval , 2011, IEEE Transactions on Multimedia.

[3]  Timothy N. Rubin,et al.  Statistical topic models for multi-label document classification , 2011, Machine Learning.

[4]  Yihong Gong,et al.  Multi-labelled classification using maximum entropy method , 2005, SIGIR '05.

[5]  Yuhong Guo,et al.  Multi-Label Classification Using Conditional Dependency Networks , 2011, IJCAI.

[6]  Volker Tresp,et al.  Multi-label informed latent semantic indexing , 2005, SIGIR '05.

[7]  Grigorios Tsoumakas,et al.  Multilabel Text Classification for Automated Tag Suggestion , 2008 .

[8]  Grigorios Tsoumakas,et al.  Random k -Labelsets: An Ensemble Method for Multilabel Classification , 2007, ECML.

[9]  Geoff Holmes,et al.  Classifier chains for multi-label classification , 2009, Machine Learning.

[10]  Tao Mei,et al.  Correlative multi-label video annotation , 2007, ACM Multimedia.

[11]  Eyke Hüllermeier,et al.  Label ranking by learning pairwise preferences , 2008, Artif. Intell..

[12]  Lukasz A. Kurgan,et al.  Multi-label associative classification of medical documents from MEDLINE , 2005, Fourth International Conference on Machine Learning and Applications (ICMLA'05).

[13]  Kivanc M. Ozonat,et al.  Towards a universal marketplace over the web: statistical multi-label classification of service provider forms with simulated annealing , 2009, KDD.

[14]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[15]  Marcel Worring,et al.  The challenge problem for automated detection of 101 semantic concepts in multimedia , 2006, MM '06.

[16]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[17]  Grigorios Tsoumakas,et al.  Multi-Label Classification of Music into Emotions , 2008, ISMIR.

[18]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[19]  Chris H. Q. Ding,et al.  Multi-Label Classification: Inconsistency and Class Balanced K-Nearest Neighbor , 2010, AAAI.

[20]  Dejan Gjorgjevikj,et al.  Efficient Two Stage Voting Architecture for Pairwise Multi-label Classification , 2010, Australasian Conference on Artificial Intelligence.

[21]  Jieping Ye,et al.  A shared-subspace learning framework for multi-label classification , 2010, TKDD.

[22]  Lei Wu,et al.  Lift: Multi-Label Learning with Label-Specific Features , 2015, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Peter Wiemer-Hastings,et al.  Latent semantic analysis , 2004, Annu. Rev. Inf. Sci. Technol..

[24]  Concha Bielza,et al.  Multi-dimensional classification with Bayesian networks , 2011, Int. J. Approx. Reason..

[25]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[26]  Jieping Ye,et al.  Extracting shared subspace for multi-label classification , 2008, KDD.

[27]  Kun Zhang,et al.  Multi-label learning by exploiting label dependency , 2010, KDD.

[28]  M. Craven,et al.  Pairwise learning of multilabel classifications with perceptrons , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[29]  Yoram Singer,et al.  BoosTexter: A Boosting-based System for Text Categorization , 2000, Machine Learning.

[30]  Grigorios Tsoumakas,et al.  Random K-labelsets for Multilabel Classification , 2022 .

[31]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[32]  Jason Weston,et al.  A kernel method for multi-labelled classification , 2001, NIPS.

[33]  Eyke Hüllermeier,et al.  Multilabel classification via calibrated label ranking , 2008, Machine Learning.

[34]  Eyke Hüllermeier,et al.  Combining instance-based learning and logistic regression for multilabel classification , 2009, Machine Learning.

[35]  Saso Dzeroski,et al.  An extensive experimental comparison of methods for multi-label learning , 2012, Pattern Recognit..

[36]  Allison Petrosino,et al.  Using information gain to build meaningful decision forests for multilabel classification , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[37]  Grigorios Tsoumakas,et al.  MULAN: A Java Library for Multi-Label Learning , 2011, J. Mach. Learn. Res..

[38]  Rong Yan,et al.  Model-shared subspace boosting for multi-label classification , 2007, KDD '07.

[39]  Naonori Ueda,et al.  Parametric Mixture Models for Multi-Labeled Text , 2002, NIPS.

[40]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[41]  François Pachet,et al.  Improving Multilabel Analysis of Music Titles: A Large-Scale Validation of the Correction Approach , 2009, IEEE Transactions on Audio, Speech, and Language Processing.

[42]  Rémi Gilleron,et al.  Learning Multi-label Alternating Decision Trees from Texts and Data , 2003, MLDM.

[43]  Tat-Seng Chua,et al.  Automatic image annotation via local multi-label classification , 2008, CIVR '08.

[44]  Remo Guidieri.  Res , 1995, RES: Anthropology and Aesthetics.

[45]  Eyke Hüllermeier,et al.  Dependent binary relevance models for multi-label classification , 2014, Pattern Recognit..

[46]  Andrew McCallum,et al.  Collective multi-label classification , 2005, CIKM '05.

[47]  Amanda Clare,et al.  Knowledge Discovery in Multi-label Phenotype Data , 2001, PKDD.

[48]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[49]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[50]  Charles Elkan,et al.  Learning and Inference in Probabilistic Classifier Chains with Beam Search , 2012, ECML/PKDD.

[51]  Yiming Yang,et al.  A Comparative Study on Feature Selection in Text Categorization , 1997, ICML.

[52]  Xian-Sheng Hua,et al.  A transductive multi-label learning approach for video concept detection , 2011, Pattern Recognit..

[53]  Yiming Yang,et al.  Multilabel classification with meta-level features in a learning-to-rank framework , 2011, Machine Learning.

[54]  O. J. Dunn.  Multiple Comparisons among Means , 1961 .

[55]  Robert E. Schapire,et al.  Hierarchical multi-label prediction of gene function , 2006, Bioinform..

[56]  Zhi-Hua Zhou,et al.  Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization , 2006, IEEE Transactions on Knowledge and Data Engineering.

[57]  Alexandre Bernardino,et al.  Matrix Completion for Multi-label Image Classification , 2011, NIPS.

[58]  Nicolò Cesa-Bianchi,et al.  Synergy of multi-label hierarchical ensembles, data fusion, and cost-sensitive methods for gene functional inference , 2012, Machine Learning.

[59]  Geoff Holmes,et al.  Multi-label Classification Using Ensembles of Pruned Sets , 2008, 2008 Eighth IEEE International Conference on Data Mining.