Shared subspace least squares multi-label linear discriminant analysis

Multi-label linear discriminant analysis (MLDA) has been explored for multi-label dimension reduction. However, MLDA involves dense matrices eigen-decomposition which is known to be computationally expensive for large-scale problems. In this paper, we show that the formulation of MLDA can be equivalently casted as a least squares problem so as to significantly reduce the computation burden and scale to the data collections with higher dimension. Further, it is also found that appealing regularization techniques can be incorporated into the least-squares model to boost generalization accuracy. Experimental results on several popular multi-label benchmarks not only verify the established equivalence relationship, but also demonstrate the effectiveness and efficiency of our proposed algorithms.

[1]  Mark J. Huiskes,et al.  The MIR flickr retrieval evaluation , 2008, MIR '08.

[2]  Xiaobo Zhou,et al.  Nonconvex Penalty Based Low-Rank Representation and Sparse Regression for eQTL Mapping , 2017, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[3]  Rémi Gilleron,et al.  Learning Multi-label Alternating Decision Trees from Texts and Data , 2003, MLDM.

[4]  Elias Oliveira,et al.  Multi-label incremental learning applied to web page categorization , 2013, Neural Computing and Applications.

[5]  Tat-Seng Chua,et al.  NUS-WIDE: a real-world web image database from National University of Singapore , 2009, CIVR '09.

[6]  Heng Huang,et al.  From Protein Sequence to Protein Function via Multi-Label Linear Discriminant Analysis , 2017, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[7]  Jason Weston,et al.  A kernel method for multi-labelled classification , 2001, NIPS.

[8]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[9]  Shuiwang Ji,et al.  SLEP: Sparse Learning with Efficient Projections , 2011 .

[10]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[11]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[12]  Jieping Ye,et al.  Least squares linear discriminant analysis , 2007, ICML '07.

[13]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[14]  Jieping Ye,et al.  Canonical Correlation Analysis for Multilabel Classification: A Least-Squares Formulation, Extensions, and Analysis , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Liang Tao,et al.  A least squares formulation of multi-label linear discriminant analysis , 2015, Neurocomputing.

[16]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[17]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[18]  Chih-Jen Lin,et al.  Probability Estimates for Multi-class Classification by Pairwise Coupling , 2003, J. Mach. Learn. Res..

[19]  Tao Zhang,et al.  Semi-supervised dictionary learning via local sparse constraints for violence detection , 2017, Pattern Recognit. Lett..

[20]  Daniel Gatica-Perez,et al.  Modeling Semantic Aspects for Cross-Media Image Indexing , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Robert E. Schapire,et al.  Hierarchical multi-label prediction of gene function , 2006, Bioinform..

[22]  Zhi-Hua Zhou,et al.  Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization , 2006, IEEE Transactions on Knowledge and Data Engineering.

[23]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[24]  Koby Crammer,et al.  A Family of Additive Online Algorithms for Category Ranking , 2003, J. Mach. Learn. Res..

[25]  Grigorios Tsoumakas,et al.  An Empirical Study of Lazy Multilabel Classification Algorithms , 2008, SETN.

[26]  Jiawei Han,et al.  SRDA: An Efficient Algorithm for Large-Scale Discriminant Analysis , 2008, IEEE Transactions on Knowledge and Data Engineering.

[27]  Sunho Park,et al.  Max-margin embedding for multi-label learning , 2013, Pattern Recognit. Lett..

[28]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[29]  Yoram Singer,et al.  BoosTexter: A Boosting-based System for Text Categorization , 2000, Machine Learning.

[30]  Mingxuan Sun,et al.  A Deep Multi-Modal CNN for Multi-Instance Multi-Label Image Classification , 2018, IEEE Transactions on Image Processing.

[31]  Przemyslaw Kazienko,et al.  Relational large scale multi-label classification method for video categorization , 2012, Multimedia Tools and Applications.

[32]  Nicu Sebe,et al.  Web Image Annotation Via Subspace-Sparsity Collaborated Feature Selection , 2012, IEEE Transactions on Multimedia.

[33]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[34]  Nenghai Yu,et al.  Learning Spatial Regularization with Image-Level Supervisions for Multi-label Image Classification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Naonori Ueda,et al.  Parametric Mixture Models for Multi-Labeled Text , 2002, NIPS.

[36]  Chris H. Q. Ding,et al.  Multi-label Linear Discriminant Analysis , 2010, ECCV.

[37]  Volker Roth,et al.  Improved functional prediction of proteins by learning kernel combinations in multilabel settings , 2007, BMC Bioinformatics.

[38]  Ivor W. Tsang,et al.  Discovering Low-Rank Shared Concept Space for Adapting Text Mining Models , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Johannes Fürnkranz,et al.  Round Robin Classification , 2002, J. Mach. Learn. Res..

[40]  Gene H. Golub,et al.  Matrix computations , 1983 .

[41]  John Langford,et al.  Multi-Label Prediction via Compressed Sensing , 2009, NIPS.

[42]  Xiangjian He,et al.  Discriminative Dictionary Learning With Motion Weber Local Descriptor for Violence Detection , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[43]  Lei Tang,et al.  Large scale multi-label classification via metalabeler , 2009, WWW '09.

[44]  Tommy W. S. Chow,et al.  A general soft label based Linear Discriminant Analysis for semi-supervised dimensionality reduction , 2014, Neural Networks.

[45]  Grigorios Tsoumakas,et al.  Multi-Label Classification of Music into Emotions , 2008, ISMIR.

[46]  Zhi-Hua Zhou,et al.  Multilabel dimensionality reduction via dependence maximization , 2008, TKDD.

[47]  Hsuan-Tien Lin,et al.  Multilabel Classification with Principal Label Space Transformation , 2012, Neural Computation.

[48]  Cheong Hee Park,et al.  On applying linear discriminant analysis for multi-labeled problems , 2008, Pattern Recognit. Lett..

[49]  Jieping Ye,et al.  Hypergraph spectral learning for multi-label classification , 2008, KDD.

[50]  Jieping Ye,et al.  A shared-subspace learning framework for multi-label classification , 2010, TKDD.

[51]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[52]  Yu Liu,et al.  CNN-RNN: a large-scale hierarchical image classification framework , 2018, Multimedia Tools and Applications.

[53]  Piotr Synak,et al.  Multi-Label Classification of Emotions in Music , 2006, Intelligent Information Systems.