Generative attention adversarial classification network for unsupervised domain adaptation

Abstract Domain adaptation is a significant and popular issue of solving distribution discrepancy among different domains in computer vision. Generally, previous works proposed are mainly devoted to reducing domain shift between source domain with labeled data and target domain without labels. Adversarial learning in deep networks has already been widely applied to learn disentangled and transferable features between two different domains to minimize domains distribution discrepancy. However, these methods rarely consider class distributions among source data during adversarial learning, and they pay little attention to these transferable regions among source and target domains images. In this paper, we propose a Generative Attention Adversarial Classification Network (GAACN) model for unsupervised domain adaptation. To learn a joint feature distribution between source and target domains, we present an improved generative adversarial network (GAN) following the feature extractor. Firstly, the discriminator of GAN discriminates the distribution of domains and the classes distribution among source data during adversarial learning, so that our feature extractor can learn a joint feature distribution between source and target domains and maintain the classes consistent simultaneously. Secondly, we present an attention module embedded in GAN, which allows the discriminator to discriminate the transferable regions among the images of source and target domains. Lastly, we propose a simple and efficient method which allocates pseudo-labels for unlabeled target data, and it can improve the performance of our model GAACN while mitigating negative transfer. Extensive experiments demonstrate that our proposed model achieves perfect results on several standard domain adaptation datasets.

[1]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[2]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[3]  Barbara Plank,et al.  Strong Baselines for Neural Semi-Supervised Learning under Domain Shift , 2018, ACL.

[4]  Huchuan Lu,et al.  Hyperfusion-Net: Hyper-densely reflective feature fusion for salient object detection , 2019, Pattern Recognit..

[5]  Yuan Shi,et al.  Geodesic flow kernel for unsupervised domain adaptation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Erik Cambria,et al.  Recent Trends in Deep Learning Based Natural Language Processing , 2017, IEEE Comput. Intell. Mag..

[7]  Venu Govindaraju,et al.  Domain adaptive representation learning for facial action unit recognition , 2020, Pattern Recognit..

[8]  Kate Saenko,et al.  Synthetic to Real Adaptation with Generative Correlation Alignment Networks , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[9]  Yan Zhang,et al.  Deep conditional adaptation networks and label correlation transfer for unsupervised domain adaptation , 2020, Pattern Recognit..

[10]  Mingkui Tan,et al.  Domain-Symmetric Networks for Adversarial Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Patrick Pérez,et al.  ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Yoshua Bengio,et al.  Semi-supervised Learning by Entropy Minimization , 2004, CAP.

[13]  Chen Zhang,et al.  Semi-supervised domain adaptation via Fredholm integral based kernel methods , 2019, Pattern Recognit..

[14]  Zhiguo Cao,et al.  Two-dimensional subspace alignment for convolutional activations adaptation , 2017, Pattern Recognit..

[15]  Taesung Park,et al.  CyCADA: Cycle-Consistent Adversarial Domain Adaptation , 2017, ICML.

[16]  Xiaogang Wang,et al.  Residual Attention Network for Image Classification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Michael I. Jordan,et al.  Conditional Adversarial Domain Adaptation , 2017, NeurIPS.

[18]  Han Zhang,et al.  Self-Attention Generative Adversarial Networks , 2018, ICML.

[19]  Junbo Wang,et al.  Learning visual relationship and context-aware attention for image captioning , 2020, Pattern Recognit..

[20]  Mengjie Zhang,et al.  Deep Reconstruction-Classification Networks for Unsupervised Domain Adaptation , 2016, ECCV.

[21]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[22]  Jianmin Wang,et al.  Multi-Adversarial Domain Adaptation , 2018, AAAI.

[23]  Pong C. Yuen,et al.  Learning domain-shared group-sparse representation for unsupervised domain adaptation , 2018, Pattern Recognit..

[24]  Yang Zou,et al.  Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training , 2018, ArXiv.

[25]  Trevor Darrell,et al.  LSDA: Large Scale Detection through Adaptation , 2014, NIPS.

[26]  Qiang Yang,et al.  Boosting for transfer learning , 2007, ICML '07.

[27]  Tao Mei,et al.  Learning Multi-attention Convolutional Neural Network for Fine-Grained Image Recognition , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[28]  Dumitru Erhan,et al.  Deep Neural Networks for Object Detection , 2013, NIPS.

[29]  Nicu Sebe,et al.  Unsupervised Domain Adaptation Using Feature-Whitening and Consensus Loss , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Dacheng Tao,et al.  Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation , 2019, NeurIPS.

[31]  Ming-Yu Liu,et al.  Coupled Generative Adversarial Networks , 2016, NIPS.

[32]  Hans-Peter Kriegel,et al.  Integrating structured biological data by Kernel Maximum Mean Discrepancy , 2006, ISMB.

[33]  Yue Cao,et al.  Transferable Representation Learning with Deep Adaptation Networks , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Michael I. Jordan,et al.  Unsupervised Domain Adaptation with Residual Transfer Networks , 2016, NIPS.

[35]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[36]  Mengjie Zhang,et al.  Domain Generalization for Object Recognition with Multi-task Autoencoders , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[37]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[38]  Philip S. Yu,et al.  Visual Domain Adaptation with Manifold Embedded Distribution Alignment , 2018, ACM Multimedia.

[39]  Ming-Hsuan Yang,et al.  Learning to Adapt Structured Output Space for Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[40]  Trevor Darrell,et al.  Adapting Visual Category Models to New Domains , 2010, ECCV.

[41]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Douglas R. Heisterkamp,et al.  Adapting instance weights for unsupervised domain adaptation using quadratic mutual information and subspace learning , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[43]  Yi Yang,et al.  Contrastive Adaptation Network for Unsupervised Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Zongyuan Ge,et al.  Training data independent image registration using generative adversarial networks and domain adaptation , 2020, Pattern Recognit..

[45]  Kate Saenko,et al.  Return of Frustratingly Easy Domain Adaptation , 2015, AAAI.

[46]  Kate Saenko,et al.  Deep CORAL: Correlation Alignment for Deep Domain Adaptation , 2016, ECCV Workshops.

[47]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[48]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Jun Fu,et al.  Hierarchically Supervised Deconvolutional Network for Semantic Video Segmentation , 2017, Pattern Recognit..

[50]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[51]  Luc Van Gool,et al.  ROAD: Reality Oriented Adaptation for Semantic Segmentation of Urban Scenes , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[52]  Carlos D. Castillo,et al.  Generate to Adapt: Aligning Domains Using Generative Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[53]  Jan Kautz,et al.  Unsupervised Image-to-Image Translation Networks , 2017, NIPS.

[54]  Ricardo da Silva Torres,et al.  Semi-supervised transfer subspace for domain adaptation , 2018, Pattern Recognit..

[55]  Michael I. Jordan,et al.  Deep Transfer Learning with Joint Adaptation Networks , 2016, ICML.

[56]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Rama Chellappa,et al.  Domain adaptation for object recognition: An unsupervised approach , 2011, 2011 International Conference on Computer Vision.

[58]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[59]  Sethuraman Panchanathan,et al.  Deep Hashing Network for Unsupervised Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.