Penalized partial least squares for multi-label data

Multi-label learning has attracted an increasing attention from many domains, because of its great potential applications. Although many learning methods have been witnessed, two major challenges are still not handled very well. They are the correlations and the high dimensionality of data. In this paper, we exploit the inherent property of the multi-label data and propose an effective sparse multi-label learning algorithm. Specifically, it handles the high-dimensional multi-label data by using a regularized partial least squares discriminant analysis with a l1-norm penalty. Consequently, the proposed method can not only capture the label correlations effectively, but also perform the operation of dimensionality reduction at the same time. The experimental results conducted on eight public data sets show that our method is promising and outperformed the state-of-the-art multi-label classifiers in most cases.

[1]  Yang Song,et al.  Automatic tag recommendation algorithms for social recommender systems , 2011, ACM Trans. Web.

[2]  Grigorios Tsoumakas,et al.  An Empirical Study of Lazy Multilabel Classification Algorithms , 2008, SETN.

[3]  S. D. Jong.  SIMPLS: an alternative approach to partial least squares regression , 1993 .

[4]  Shou-De Lin,et al.  Cost-Sensitive Multi-Label Learning for Audio Tag Annotation and Retrieval , 2011, IEEE Transactions on Multimedia.

[5]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[6]  Eyke Hüllermeier,et al.  Multilabel classification via calibrated label ranking , 2008, Machine Learning.

[7]  Chien-Li Chou,et al.  Effective Semantic Annotation by Image-to-Concept Distribution Model , 2011, IEEE Transactions on Multimedia.

[8]  Eyke Hüllermeier,et al.  Combining instance-based learning and logistic regression for multilabel classification , 2009, Machine Learning.

[9]  Zhi-Hua Zhou,et al.  Multilabel dimensionality reduction via dependence maximization , 2008, TKDD.

[10]  Juho Rousu,et al.  Kernel-Based Learning of Hierarchical Multilabel Classification Models , 2006, J. Mach. Learn. Res..

[11]  Sushant Sachdeva,et al.  Dimension Reduction , 2008, Encyclopedia of GIS.

[12]  Jieping Ye,et al.  Canonical Correlation Analysis for Multilabel Classification: A Least-Squares Formulation, Extensions, and Analysis , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Grigorios Tsoumakas,et al.  Random K-labelsets for Multilabel Classification , 2022 .

[14]  Wynne W. Chin,et al.  New Perspectives in Partial Least Squares and Related Methods , 2013 .

[15]  Zhi-Hua Zhou,et al.  Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization , 2006, IEEE Transactions on Knowledge and Data Engineering.

[16]  Jesse Read,et al.  Scalable Multi-label Classification , 2010 .

[17]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[18]  Anne-Laure Boulesteix,et al.  Partial least squares: a versatile tool for the analysis of high-dimensional genomic data , 2006, Briefings Bioinform..

[19]  Grigorios Tsoumakas,et al.  Correlation-Based Pruning of Stacked Binary Relevance Models for Multi-Label Learning , 2009 .

[20]  John Langford,et al.  Multi-Label Prediction via Compressed Sensing , 2009, NIPS.

[21]  Lior Rokach,et al.  Data Mining And Knowledge Discovery Handbook , 2005 .

[22]  James T. Kwok,et al.  MultiLabel Classification on Tree- and DAG-Structured Hierarchies , 2011, ICML.

[23]  Guandong Xu.  2014 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2014, Beijing, China, August 17-20, 2014 , 2014 .

[24]  Grigorios Tsoumakas,et al.  MULAN: A Java Library for Multi-Label Learning , 2011, J. Mach. Learn. Res..

[25]  ZhouZhi-Hua,et al.  Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization , 2006 .

[26]  Jieping Ye,et al.  A shared-subspace learning framework for multi-label classification , 2010, TKDD.

[27]  Michael K. Ng,et al.  Sparse Canonical Correlation Analysis: New Formulation and Algorithm , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Giorgio Valentini,et al.  True Path Rule Hierarchical Ensembles for Genome-Wide Gene Function Prediction , 2011, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[29]  S. Keleş,et al.  Sparse partial least squares regression for simultaneous dimension reduction and variable selection , 2010, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[30]  Hsuan-Tien Lin,et al.  Multilabel Classification with Principal Label Space Transformation , 2012, Neural Computation.

[31]  Xindong Wu,et al.  MLSLR: Multilabel Learning via Sparse Logistic Regression , 2014, Inf. Sci..

[32]  Philippe Besse,et al.  Sparse PLS discriminant analysis: biologically relevant feature selection and graphical displays for multiclass problems , 2011, BMC Bioinformatics.

[33]  Youngjo Lee,et al.  Sparse partial least-squares regression and its applications to high-throughput data analysis , 2011 .

[34]  Xindong Wu,et al.  A NEW SUPERVISED FEATURE SELECTION METHOD FOR PATTERN CLASSIFICATION , 2014, Comput. Intell..

[35]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..