Dual Perspective of Label-Specific Feature Learning for Multi-Label Classification

Label-specific features serve as an effective strategy to facilitate multi-label classification, which account for the distinct discriminative properties of each class label via tailoring its own features. Existing approaches implement this strategy in a quite straightforward way, i.e. finding the most pertinent and discriminative features for each class label and directly inducing classifiers on constructed label-specific features. In this paper, we propose a dual perspective for label-specific feature learning, where label-specific discriminative properties are considered by identifying each label’s own non-informative features and making the discrimination process immutable to variations of these features. To instantiate it, we present a perturbation-based approach D ELA to provide classifiers with label-specific immutability on simultaneously identified non-informative features, which is optimized towards a probabilistically-relaxed expected risk minimization problem. Comprehensive experiments on 10 benchmark data sets show that our approach outperforms the state-of-the-art counter-parts.

[1]  Min-Ling Zhang,et al.  Collaborative Learning of Label Semantics and Deep Label-Specific Features for Multi-Label Classification , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Min-Ling Zhang,et al.  Multi-Label Classification With Label-Specific Feature Generation: A Wrapped Approach , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Ivor W. Tsang,et al.  The Emerging Trends of Multi-Label Learning , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Hefeng Wu,et al.  Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognition , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Yuefeng Chen,et al.  AdvDrop: Adversarial Attack to DNNs by Dropping Information , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[6]  Yoshua Bengio,et al.  Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization , 2021, ArXiv.

[7]  Sunghoon Im,et al.  DRANet: Disentangling Representation and Adaptation Networks for Unsupervised Cross-Domain Adaptation , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Miao Xu,et al.  Learning from group supervision: the impact of supervision deficiency on multi-label learning , 2021, Science China Information Sciences.

[9]  Zhanshan Li,et al.  Multi-label learning with label-specific features via weighting and label entropy guided clustering ensemble , 2021, Neurocomputing.

[10]  Manik Varma,et al.  SiameseXML: Siamese Networks meet Extreme Classifiers with 100M Labels , 2021, ICML.

[11]  Min-Ling Zhang,et al.  Compositional metric learning for multi-label classification , 2020, Frontiers of Computer Science.

[12]  Aidong Zhang,et al.  Correlation Networks for Extreme Multi-label Text Classification , 2020, KDD.

[13]  Carla P. Gomes,et al.  Disentangled Variational Autoencoder based Multi-Label Classification with Covariance-Aware Multivariate Probit Model , 2020, IJCAI.

[14]  Xiu-Yi Jia,et al.  Joint Label-Specific Features and Correlation Information for Multi-Label Learning , 2020, Journal of Computer Science and Technology.

[15]  Chin-Ling Chen,et al.  Non-sparse label specific features selection for multi-label classification , 2020, Neurocomputing.

[16]  Kush R. Varshney,et al.  Invariant Risk Minimization Games , 2020, ICML.

[17]  Sid Ying-Ze Bao,et al.  Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classification , 2019, AAAI.

[18]  Joost van de Weijer,et al.  Orderless Recurrent Models for Multi-Label Classification , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Quoc V. Le,et al.  AutoAugment: Learning Augmentation Strategies From Data , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Korris Fu-Lai Chung,et al.  Leveraging Label-Specific Discriminant Mapping Features for Multi-Label Learning , 2019, ACM Trans. Knowl. Discov. Data.

[21]  Jia Zhang,et al.  Multi-label learning with label-specific features by resolving label correlations , 2018, Knowl. Based Syst..

[22]  Rodrigo C. Barros,et al.  Hierarchical Multi-Label Classification Networks , 2018, ICML.

[23]  Rob Brekelmans,et al.  Invariant Representations without Adversarial Training , 2018, NeurIPS.

[24]  Qingming Huang,et al.  Joint Feature Selection and Classification for Multilabel Learning , 2018, IEEE Transactions on Cybernetics.

[25]  Shunxiang Wu,et al.  Multi-label learning based on label-specific features and local pairwise label correlation , 2018, Neurocomputing.

[26]  Aleksander Madry,et al.  Towards Deep Learning Models Resistant to Adversarial Attacks , 2017, ICLR.

[27]  Zhi-Hua Zhou,et al.  Multi-Label Learning with Global and Local Label Correlation , 2017, IEEE Transactions on Knowledge and Data Engineering.

[28]  Stefano Soatto,et al.  Information Dropout: Learning Optimal Representations Through Noisy Computation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Wang Zhan,et al.  Multi-label Learning with Label-Specific Features via Clustering Ensemble , 2017, 2017 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[30]  Graham W. Taylor,et al.  Improved Regularization of Convolutional Neural Networks with Cutout , 2017, ArXiv.

[31]  Xiaoming Liu,et al.  Disentangled Representation Learning GAN for Pose-Invariant Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Andrea Vedaldi,et al.  Interpretable Explanations of Black Boxes by Meaningful Perturbation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[33]  Yu-Chiang Frank Wang,et al.  Learning Deep Latent Spaces for Multi-Label Classification , 2017, ArXiv.

[34]  Alexander A. Alemi,et al.  Deep Variational Information Bottleneck , 2017, ICLR.

[35]  Ben Poole,et al.  Categorical Reparameterization with Gumbel-Softmax , 2016, ICLR.

[36]  Yee Whye Teh,et al.  The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables , 2016, ICLR.

[37]  Yexiang Xue,et al.  Deep Multi-species Embedding , 2016, IJCAI.

[38]  Wei Xu,et al.  CNN-RNN: A Unified Framework for Multi-label Image Classification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Kilian Q. Weinberger,et al.  Deep Networks with Stochastic Depth , 2016, ECCV.

[40]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[41]  Xindong Wu,et al.  Learning Label Specific Features for Multi-label Classification , 2015, 2015 IEEE International Conference on Data Mining.

[42]  Xiao Li,et al.  Multi-label learning with discriminative features for each label , 2015, Neurocomputing.

[43]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[44]  Min-Ling Zhang,et al.  Lift: Multi-Label Learning with Label-Specific Features , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Ariel D. Procaccia,et al.  Variational Dropout and the Local Reparameterization Trick , 2015, NIPS.

[46]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[47]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[48]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[49]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[50]  Naftali Tishby,et al.  The information bottleneck method , 2000, ArXiv.

[51]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[52]  Pietro Perona,et al.  Overcomplete steerable pyramid filters and rotation invariance , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[53]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .