Fast Label Enhancement for Label Distribution Learning

Label Distribution Learning (LDL) has attracted increasing research attentions due to its potential to address the label ambiguity problem in machine learning and success in many real-world applications. In LDL, it is usually expensive to obtain the ground-truth label distributions of data, but it is relatively easy to obtain the logical labels of data. How to use training instances only with logical labels to learn an effective LDL model is a challenging problem. In this paper, we propose a two-step framework to address this problem. Specifically, we first design an efficient recovery model to recover the latent label distributions of training instances, named Fast Label Enhancement (FLE). Our idea is to use non-negative matrix factorization (NMF) to mine the label distribution information from the feature space. Moreover, we take the instance-class similarities into consideration to discover the importance of each label to training instances, which is useful for learning precise label distributions. Then, we train a predictive model for testing instances based on generated label distributions of training instances and an existing LDL method (e.g., SA-BFGS). Experimental results on fifteen benchmark datasets show the effectiveness of the proposed two-step framework and verify the superiority of FLE over several state-of-the-art approaches.

[1]  Jun Shu,et al.  Variational Label Enhancement , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Jihua Zhu,et al.  Bidirectional Loss Function for Label Enhancement and Distribution Learning , 2020, Knowl. Based Syst..

[3]  Jihua Zhu,et al.  Generalized Label Enhancement With Sample Correlations , 2020, IEEE Transactions on Knowledge and Data Engineering.

[4]  Xin Geng,et al.  Discrete Binary Coding based Label Distribution Learning , 2019, IJCAI.

[5]  Weiwei Li,et al.  Label distribution learning with label-specific features , 2019, IJCAI.

[6]  Shen Furao,et al.  Latent Semantics Encoding for Label Distribution Learning , 2019, IJCAI.

[7]  Yan Wang,et al.  Deep Differentiable Random Forests for Age Estimation , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Xin Geng,et al.  Soft Facial Landmark Detection by Label Distribution Learning , 2019, AAAI.

[9]  Xin Geng,et al.  Indoor Crowd Counting by Mixture of Gaussians Label Distribution Learning , 2019, IEEE Transactions on Image Processing.

[10]  Zechao Li,et al.  Facial Emotion Distribution Learning by Exploiting Low-Rank Label Correlations Locally , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Wenyu Liu,et al.  Structured random forest for label distribution learning , 2018, Neurocomputing.

[12]  N. Xu,et al.  Label Enhancement for Label Distribution Learning , 2018, IEEE Transactions on Knowledge and Data Engineering.

[13]  Ke Wang,et al.  Binary Coding based Label Distribution Learning , 2018, IJCAI.

[14]  Jianxin Wu,et al.  Age Estimation Using Expectation of Label Distribution Learning , 2018, IJCAI.

[15]  Xin Geng,et al.  Label Embedding Based on Multi-Scale Locality Preservation , 2018, IJCAI.

[16]  Min-Ling Zhang,et al.  Feature-Induced Labeling Information Enrichment for Multi-Label Learning , 2018, AAAI.

[17]  Zhi-Hua Zhou,et al.  Label Distribution Learning by Optimal Transport , 2018, AAAI.

[18]  Miao Xu,et al.  Incomplete Label Distribution Learning , 2017, IJCAI.

[19]  Ning Xu,et al.  Multi-label Learning with Label Enhancement , 2017, 2018 IEEE International Conference on Data Mining (ICDM).

[20]  Hong Yu,et al.  Multi-view clustering via multi-manifold regularized non-negative matrix factorization , 2017, Neural Networks.

[21]  Kai Zhao,et al.  Label Distribution Learning Forests , 2017, NIPS.

[22]  Xin Geng,et al.  Semi-Supervised Adaptive Label Distribution Learning for Facial Age Estimation , 2017, AAAI.

[23]  Xin Geng,et al.  Soft video parsing by label distribution learning , 2017, Frontiers of Computer Science.

[24]  Jufeng Yang,et al.  Learning Visual Sentiment Distributions via Augmented Conditional Probability Neural Network , 2017, AAAI.

[25]  Jianxin Wu,et al.  Deep Label Distribution Learning With Label Ambiguity , 2016, IEEE Transactions on Image Processing.

[26]  Xuan Zhang,et al.  Emotion Distribution Learning from Texts , 2016, EMNLP.

[27]  Xu Yang,et al.  Sparsity Conditional Energy Label Distribution Learning for Age Estimation , 2016, IJCAI.

[28]  Xin Geng,et al.  Logistic Boosting Regression for Label Distribution Learning , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Xin Geng,et al.  Multi-Label Manifold Learning , 2016, AAAI.

[30]  Peter Kontschieder,et al.  Deep Neural Decision Forests , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[31]  Xin Geng,et al.  Leveraging Implicit Relative Labeling-Importance Information for Effective Multi-label Learning , 2015, 2015 IEEE International Conference on Data Mining.

[32]  Xin Geng,et al.  Crowd counting in public video surveillance by label distribution learning , 2015, Neurocomputing.

[33]  Xin Geng,et al.  Emotion Distribution Recognition from Facial Expressions , 2015, ACM Multimedia.

[34]  Jianzhong Wang,et al.  Label propagation based semi-supervised non-negative matrix factorization for feature extraction , 2015, Neurocomputing.

[35]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[36]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[37]  Xin Geng,et al.  Multilabel Ranking with Inconsistent Rankers , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Xin Geng,et al.  Label Distribution Learning , 2013, 2013 IEEE 13th International Conference on Data Mining Workshops.

[39]  Zhi-Hua Zhou,et al.  Facial Age Estimation by Learning from Label Distributions , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Jiawei Han,et al.  Non-negative Matrix Factorization on Manifold , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[41]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[42]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[44]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[45]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[46]  D. Botstein,et al.  Cluster analysis and display of genome-wide expression patterns. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[47]  S. Della,et al.  A Maximum Entropy Approach to Natural Language Processing , 1983 .

[48]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..