Label Information Bottleneck for Label Enhancement

In this work, we focus on the challenging problem of Label Enhancement (LE), which aims to exactly recover label distributions from logical labels, and present a novel Label Information Bottleneck (LIB) method for LE. For the recovery process of label distributions, the label irrelevant information contained in the dataset may lead to unsatisfactory recovery performance. To address this limitation, we make efforts to excavate the essential label relevant information to improve the recovery performance. Our method formulates the LE problem as the following two joint processes: 1) learning the representation with the essential label relevant information, 2) recovering label distributions based on the learned representation. The label relevant information can be excavated based on the"bottleneck"formed by the learned representation. Significantly, both the label relevant information about the label assignments and the label relevant information about the label gaps can be explored in our method. Evaluation experiments conducted on several benchmark label distribution learning datasets verify the effectiveness and competitiveness of LIB. Our source codes are available https://github.com/qinghai-zheng/LIBLE

[1]  Yunan Lu,et al.  Label Enhancement by Maintaining Positive and Negative Label Relation , 2023, IEEE Transactions on Knowledge and Data Engineering.

[2]  Junchi Yu,et al.  Improving Subgraph Recognition with Variational Graph Information Bottleneck , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Wenxi Liu,et al.  Coarse-to-Fine Person Re-Identification with Auxiliary-Domain Classification and Second-Order Information Bottleneck , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Seungju Han,et al.  Disentangling Label Distribution for Long-tailed Visual Recognition , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Ivor W. Tsang,et al.  The Emerging Trends of Multi-Label Learning , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Jure Leskovec,et al.  Graph Information Bottleneck , 2020, NeurIPS.

[7]  Jun Shu,et al.  Variational Label Enhancement , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Yu Zhang,et al.  Label Enhancement for Label Distribution Learning via Prior Knowledge , 2020, IJCAI.

[9]  Zhongchao Shi,et al.  Label Distribution Learning on Auxiliary Label Space Graphs for Facial Expression Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  M. Hasegawa-Johnson,et al.  Unsupervised Speech Decomposition via Triple Information Bottleneck , 2020, ICML.

[11]  Jihua Zhu,et al.  Generalized Label Enhancement With Sample Correlations , 2020, IEEE Transactions on Knowledge and Data Engineering.

[12]  Jihua Zhu,et al.  Label Enhancement with Sample Correlations via Low-Rank Representation , 2020, AAAI.

[13]  Zechao Li,et al.  Label Distribution Learning with Label Correlations on Local Samples , 2019, IEEE Transactions on Knowledge and Data Engineering.

[14]  Xin Geng,et al.  Weakly Supervised Multi-Label Learning via Label Enhancement , 2019, IJCAI.

[15]  Xin Geng,et al.  Indoor Crowd Counting by Mixture of Gaussians Label Distribution Learning , 2019, IEEE Transactions on Image Processing.

[16]  Zechao Li,et al.  Facial Emotion Distribution Learning by Exploiting Low-Rank Label Correlations Locally , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  N. Xu,et al.  Label Enhancement for Label Distribution Learning , 2018, IEEE Transactions on Knowledge and Data Engineering.

[18]  Jianxin Wu,et al.  Age Estimation Using Expectation of Label Distribution Learning , 2018, IJCAI.

[19]  Rana Ali Amjad,et al.  Learning Representations for Neural Network-Based Classification Using the Information Bottleneck Principle , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Xin Geng,et al.  Multi-Label Manifold Learning , 2016, AAAI.

[21]  Xin Geng,et al.  Leveraging Implicit Relative Labeling-Importance Information for Effective Multi-label Learning , 2015, 2015 IEEE International Conference on Data Mining.

[22]  Xin Geng,et al.  Pre-release Prediction of Crowd Opinion on Movies by Label Distribution Learning , 2015, IJCAI.

[23]  Daan Wierstra,et al.  Stochastic Backpropagation and Approximate Inference in Deep Generative Models , 2014, ICML.

[24]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[25]  Xin Geng,et al.  Label Distribution Learning , 2013, 2013 IEEE 13th International Conference on Data Mining Workshops.

[26]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Günther Palm,et al.  A Study of the Robustness of KNN Classifiers Trained Using Soft Labels , 2006, ANNPR.

[28]  Zhang Yi,et al.  Fuzzy SVM with a new fuzzy membership function , 2006, Neural Computing & Applications.

[29]  Naftali Tishby,et al.  The information bottleneck method , 2000, ArXiv.

[30]  D. Botstein,et al.  Cluster analysis and display of genome-wide expression patterns. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[31]  Xin Geng,et al.  Label Distribution Learning Machine , 2021, ICML.