Multi-Label Learning via Feature and Label Space Dimension Reduction

In multi-label learning, each object belongs to multiple class labels simultaneously. In the data explosion age, the size of data is often huge, i.e., large number of instances, features and class labels. The high dimension of both the feature and label spaces has posed great challenges to multi-label learning problems, e.g., high time and memory costs. In this paper, we propose a new framework for multi-label learning with a large number of class labels and features, i.e., Multi-Label Learning via Feature and Label Space Dimension Reduction, namely MLL-FLSDR. Specifically, both the feature space and label space are reduced to low dimensional spaces respectively, in which the local structure of data points is utilized to constrain the geometrical structure on both the learned low dimensional spaces and guarantee the qualities of them. Then, an effective multi-label classifier is constructed from the low dimensional feature space to the latent label space. Last, the final prediction for new test data examples can be obtained by recovering from their prediction results in the latent label space with an encoding matrix learned in the previous stage. Extensive comparison experiments with the state-of-the-art approaches manifest the effectiveness of the proposed method MLL-FLSDR.

[1]  Jia Zhang,et al.  Multi-label learning with label-specific features by resolving label correlations , 2018, Knowl. Based Syst..

[2]  Arun K. Pujari,et al.  Multi-label classification using hierarchical embedding , 2018, Expert Syst. Appl..

[3]  Xiao Li,et al.  Robust label compression for multi-label classification , 2016, Knowl. Based Syst..

[4]  Bernhard Schölkopf,et al.  DiSMEC: Distributed Sparse Machines for Extreme Multi-label Classification , 2016, WSDM.

[5]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[6]  Yong Luo,et al.  Low-Rank Multi-View Learning in Matrix Completion for Multi-Label Image Classification , 2015, AAAI.

[7]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[8]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[9]  Sebastián Ventura,et al.  A Tutorial on Multilabel Learning , 2015, ACM Comput. Surv..

[10]  Dacheng Tao,et al.  Robust Extreme Multi-label Learning , 2016, KDD.

[11]  Manik Varma,et al.  Multi-label learning with millions of labels: recommending advertiser bid phrases for web pages , 2013, WWW.

[12]  Jianmin Wang,et al.  Multi-label Classification via Feature-aware Implicit Label Space Encoding , 2014, ICML.

[13]  Min-Ling Zhang,et al.  Lift: Multi-Label Learning with Label-Specific Features , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Yu-Chiang Frank Wang,et al.  Learning Deep Latent Spaces for Multi-Label Classification , 2017, ArXiv.

[15]  Qingming Huang,et al.  Improving multi-label classification with missing labels by learning label-specific features , 2019, Inf. Sci..

[16]  Manik Varma,et al.  Extreme Multi-label Loss Functions for Recommendation, Tagging, Ranking & Other Missing Label Applications , 2016, KDD.

[17]  Dejun Mu,et al.  Expede Herculem: Learning Multi Labels From Single Label , 2018, IEEE Access.

[18]  Qiang Ji,et al.  Multi-label Learning with Missing Labels , 2014, 2014 22nd International Conference on Pattern Recognition.

[19]  Zhiming Luo,et al.  Towards a unified multi-source-based optimization framework for multi-label learning , 2019, Appl. Soft Comput..

[20]  Arun K. Pujari,et al.  Group Preserving Label Embedding for Multi-Label Classification , 2018, Pattern Recognit..

[21]  Hsuan-Tien Lin,et al.  Feature-aware Label Space Dimension Reduction for Multi-label Classification , 2012, NIPS.

[22]  Venkatesh Balasubramanian,et al.  Slice: Scalable Linear Extreme Classifiers Trained on 100 Million Labels for Related Searches , 2019, WSDM.

[23]  Qingming Huang,et al.  Multi-label classification by exploiting local positive and negative pairwise label correlation , 2017, Neurocomputing.

[24]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[25]  Qingming Huang,et al.  Multi-View Multi-Label Learning With View-Label-Specific Features , 2019, IEEE Access.

[26]  Jianhua Xu,et al.  A multi-label feature extraction algorithm via maximizing feature variance and feature-label dependence simultaneously , 2016, Knowl. Based Syst..

[27]  Johannes Fürnkranz,et al.  Large-Scale Multi-label Text Classification - Revisiting Neural Networks , 2013, ECML/PKDD.

[28]  Qinghua Hu,et al.  Multi-label feature selection with missing labels , 2018, Pattern Recognit..

[29]  Prateek Jain,et al.  Sparse Local Embeddings for Extreme Multi-label Classification , 2015, NIPS.

[30]  Pravesh Kothari,et al.  Efficient Algorithms for Outlier-Robust Regression , 2018, COLT.

[31]  Shunxiang Wu,et al.  An Efficient Stacking Model of Multi-Label Classification Based on Pareto Optimum , 2019, IEEE Access.

[32]  Philippe Gagnon,et al.  Bayesian robustness to outliers in linear regression and ratio estimation , 2019, Brazilian Journal of Probability and Statistics.

[33]  Chin-Ling Chen,et al.  Non-sparse label specific features selection for multi-label classification , 2020, Neurocomputing.

[34]  Grigorios Tsoumakas,et al.  Mining Multi-label Data , 2010, Data Mining and Knowledge Discovery Handbook.

[35]  Francisco Charte,et al.  Multilabel Classification: Problem Analysis, Metrics and Techniques , 2016 .

[36]  Zhiming Luo,et al.  Manifold regularized discriminative feature selection for multi-label learning , 2019, Pattern Recognit..

[37]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[38]  Xindong Wu,et al.  Learning Label-Specific Features and Class-Dependent Labels for Multi-Label Classification , 2016, IEEE Transactions on Knowledge and Data Engineering.

[39]  Zhiwen Yu,et al.  A Multi-Label Learning Method Using Affinity Propagation and Support Vector Machine , 2017, IEEE Access.

[40]  Xueyan Liu,et al.  Selecting label-dependent features for multi-label classification , 2017, Neurocomputing.

[41]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[42]  Manik Varma,et al.  FastXML: a fast, accurate and stable tree-classifier for extreme multi-label learning , 2014, KDD.

[43]  Bo An,et al.  Collaboration based Multi-Label Learning , 2019, AAAI.

[44]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.