A Hierarchical-Tree-Based Method for Generative Zero-Shot Learning

It is currently a popular practice to use the class semantic information and the conditional generative adversarial network (CGAN) technique to generate visual features for the unseen classes in zero-shot learning (ZSL). However, there is currently no good ways to ensure that the generated visual features can always be beneficial to the prediction of the unseen classes. To alleviate this problem, we propose a hierarchical-tree-based method for constraining the generation process of CGAN, which can tune the generated visual features based on the multi-level class information. Moreover, to enhance the mapping ability of the model from the visual space to the semantic space, we add a multi-expert module to the traditional single mapping channel, which helps the model to mine the mapping relationship between the visual space and the semantic space. Extensive experimental results on five benchmark data sets show that our method can achieve better generalization ability than other existing generative ZSL algorithms.

[1]  David A. Forsyth,et al.  Describing objects by their attributes , 2009, CVPR.

[2]  Geoffrey E. Hinton,et al.  Zero-shot Learning with Semantic Output Codes , 2009, NIPS.

[3]  Tong Li,et al.  Conditional Wasserstein generative adversarial network-gradient penalty-based approach to alleviating imbalanced data classification , 2020, Inf. Sci..

[4]  Sudipta Mukhopadhyay,et al.  Single Image Haze Removal Using Conditional Wasserstein Generative Adversarial Networks , 2019, 2019 27th European Signal Processing Conference (EUSIPCO).

[5]  Zi Huang,et al.  Alleviating Feature Confusion for Generative Zero-shot Learning , 2019, ACM Multimedia.

[6]  Christoph H. Lampert,et al.  Learning to detect unseen object classes by between-class attribute transfer , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Andrew Zisserman,et al.  Automated Flower Classification over a Large Number of Classes , 2008, 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing.

[8]  Bernt Schiele,et al.  Learning What and Where to Draw , 2016, NIPS.

[9]  Philip H. S. Torr,et al.  An embarrassingly simple approach to zero-shot learning , 2015, ICML.

[10]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Andrew Y. Ng,et al.  Zero-Shot Learning Through Cross-Modal Transfer , 2013, NIPS.

[12]  Zhong Ming,et al.  Fuzziness-based online sequential extreme learning machine for classification problems , 2018, Soft Computing.

[13]  Cordelia Schmid,et al.  Label-Embedding for Attribute-Based Classification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Marc'Aurelio Ranzato,et al.  DeViSE: A Deep Visual-Semantic Embedding Model , 2013, NIPS.

[15]  Xizhao Wang,et al.  A review on neural networks with random weights , 2018, Neurocomputing.

[16]  Bernt Schiele,et al.  Evaluation of output embeddings for fine-grained image classification , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Chen Xu,et al.  The SUN Attribute Database: Beyond Categories for Deeper Scene Understanding , 2014, International Journal of Computer Vision.

[18]  Bernt Schiele,et al.  Gaze Embeddings for Zero-Shot Image Classification , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Christoph H. Lampert,et al.  Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Hema A. Murthy,et al.  A Generative Model for Zero Shot Learning Using Conditional Variational Autoencoders , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[21]  Xi Peng,et al.  A Generative Adversarial Approach for Zero-Shot Learning from Noisy Texts , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[22]  Shiguang Shan,et al.  Transferable Contrastive Network for Generalized Zero-Shot Learning , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[23]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[24]  Alexandros Nanopoulos,et al.  Hubs in Space: Popular Nearest Neighbors in High-Dimensional Data , 2010, J. Mach. Learn. Res..

[25]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[26]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[27]  Pietro Perona,et al.  The Caltech-UCSD Birds-200-2011 Dataset , 2011 .

[28]  Bernt Schiele,et al.  Feature Generating Networks for Zero-Shot Learning , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[30]  Gustavo Carneiro,et al.  Multi-modal Cycle-consistent Generalized Zero-Shot Learning , 2018, ECCV.