Bi-directional class-wise adversaries for unsupervised domain adaptation

Unsupervised domain adaptation relies on well-labeled auxiliary source domain information to get better performance on the unlabeled target domain. It has shown tremendous importance for various classification and segmentation problems. Classical methods rely on diminishing the domain discrepancy in the latent space but ignore class-wise information, which will lead to elimination of the inherent data structure. To avoid destroying the inherent structure during unsupervised domain adaptation, we propose a Bi-Directional Class-level Adversaries cross-domain model (BDCA) with two symmetric classifiers interpolating two latent spaces to build a tunnel between the source domain and target domain. Specifically, we propose a class-level discrepancy metric to enforce domain consistency during the trend of domain adaption. We also employ two symmetric classifiers that are collectively optimized to maximize the discrepancy on target sample prediction. Extensive experiments are conducted on four publicly available datasets (i.e. office-31, office-home, GTAV and Cityscapes) and two challenging computer vision prediction problems, i.e., image classification and semantic segmentation. Quantitative and qualitative results demonstrate the effectiveness of our proposed model.

[1]  Tatsuya Harada,et al.  Asymmetric Tri-training for Unsupervised Domain Adaptation , 2017, ICML.

[2]  Yun Fu,et al.  Robust Transfer Metric Learning for Image Classification , 2017, IEEE Transactions on Image Processing.

[3]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[4]  Mingli Ding,et al.  Bi-Directional Generation for Unsupervised Domain Adaptation , 2020, AAAI.

[5]  Ruxin Chen,et al.  Temporal Attentive Alignment for Large-Scale Video Domain Adaptation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[6]  Pedro H. O. Pinheiro,et al.  Unsupervised Domain Adaptation with Similarity Learning , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  Qingming Huang,et al.  Gradually Vanishing Bridge for Adversarial Domain Adaptation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Yi Yang,et al.  Taking a Closer Look at Domain Shift: Category-Level Adversaries for Semantics Consistent Domain Adaptation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Nicu Sebe,et al.  Unsupervised Domain Adaptation Using Feature-Whitening and Consensus Loss , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Taesung Park,et al.  CyCADA: Cycle-Consistent Adversarial Domain Adaptation , 2017, ICML.

[11]  Sethuraman Panchanathan,et al.  Deep Hashing Network for Unsupervised Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Antonio M. López,et al.  The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Hamido Fujita,et al.  Inverse projection group sparse representation for tumor classification: A low rank variation dictionary approach , 2020, Knowl. Based Syst..

[14]  Barbara Caputo,et al.  AdaGraph: Unifying Predictive and Continuous Domain Adaptation Through Graphs , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[16]  Shuicheng Yan,et al.  Hybrid CNN and Dictionary-Based Models for Scene Recognition and Domain Adaptation , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Vladlen Koltun,et al.  Playing for Data: Ground Truth from Computer Games , 2016, ECCV.

[18]  Yun Fu,et al.  Deep Transfer Low-Rank Coding for Cross-Domain Learning , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[20]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Jianmin Wang,et al.  Multi-Adversarial Domain Adaptation , 2018, AAAI.

[22]  Junzhou Huang,et al.  Progressive Feature Alignment for Unsupervised Domain Adaptation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Hai He,et al.  Unsupervised Domain Adaptation via Importance Sampling , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[25]  Rui Xu,et al.  Deep Joint Depth Estimation and Color Correction From Monocular Underwater Images Based on Unsupervised Adaptation Networks , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[26]  Qingming Huang,et al.  Towards Discriminability and Diversity: Batch Nuclear-Norm Maximization Under Label Insufficient Situations , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Ming-Hsuan Yang,et al.  Learning to Adapt Structured Output Space for Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[28]  Patrick Pérez,et al.  ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Xiaobing Zhang,et al.  Contextual-Relation Consistent Domain Adaptation for Semantic Segmentation , 2020, ECCV.

[30]  Dumitru Erhan,et al.  Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Dong Liu,et al.  Fully Convolutional Adaptation Networks for Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[32]  Luc Van Gool,et al.  DLOW: Domain Flow for Adaptation and Generalization , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Carlos D. Castillo,et al.  Generate to Adapt: Aligning Domains Using Generative Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[34]  Jianmin Wang,et al.  Transferability vs. Discriminability: Batch Spectral Penalization for Adversarial Domain Adaptation , 2019, ICML.

[35]  Xingyuan Wang,et al.  New strategy for CBIR by combining low-level visual features with a colour descriptor , 2019, IET Image Process..

[36]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[37]  Chenglin Wen,et al.  Deep learning fault diagnosis method based on global optimization GAN for unbalanced data , 2020, Knowl. Based Syst..

[38]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Chuan Zhang,et al.  Ternary radial harmonic Fourier moments based robust stereo image zero-watermarking algorithm , 2019, Inf. Sci..

[40]  Yu Wang,et al.  A decisive content based image retrieval approach for feature fusion in visual and textual images , 2019, Knowl. Based Syst..

[41]  Kate Saenko,et al.  Return of Frustratingly Easy Domain Adaptation , 2015, AAAI.

[42]  Shijian Lu,et al.  Scale variance minimization for unsupervised domain adaptation in image segmentation , 2021, Pattern Recognit..

[43]  Chen-Yu Lee,et al.  Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Philip S. Yu,et al.  Adaptation Regularization: A General Framework for Transfer Learning , 2014, IEEE Transactions on Knowledge and Data Engineering.

[46]  Ming-Hsuan Yang,et al.  CrDoCo: Pixel-Level Domain Transfer With Cross-Domain Consistency , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Chuan Zhang,et al.  Detected text-based image retrieval approach for textual images , 2019, IET Image Process..

[48]  Xingyuan Wang,et al.  The method for image retrieval based on multi-factors correlation utilizing block truncation coding , 2014, Pattern Recognit..

[49]  Meng Wang,et al.  Cross-Entropy Adversarial View Adaptation for Person Re-Identification , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[50]  Xingyuan Wang,et al.  Visual and textual information fusion using Kernel method for content based image retrieval , 2018, Inf. Fusion.

[51]  Jianmin Wang,et al.  Transferable Attention for Domain Adaptation , 2019, AAAI.

[52]  Chuan-Xian Ren,et al.  Enhanced Transport Distance for Unsupervised Domain Adaptation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Xingyuan Wang,et al.  A novel method for image retrieval based on structure elements' descriptor , 2013, J. Vis. Commun. Image Represent..

[54]  Chong-Wah Ngo,et al.  Semi-supervised Domain Adaptation with Subspace Learning for visual recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Yuan Shi,et al.  Geodesic flow kernel for unsupervised domain adaptation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Trevor Darrell,et al.  Adapting Visual Category Models to New Domains , 2010, ECCV.

[57]  Tatsuya Harada,et al.  Maximum Classifier Discrepancy for Unsupervised Domain Adaptation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[58]  Michael I. Jordan,et al.  Deep Transfer Learning with Joint Adaptation Networks , 2016, ICML.

[59]  Mingkui Tan,et al.  Domain-Symmetric Networks for Adversarial Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Vinay P. Namboodiri,et al.  Attending to Discriminative Certainty for Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[61]  Xing-Yuan Wang,et al.  An effective method for color image retrieval based on texture , 2012, Comput. Stand. Interfaces.

[62]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[63]  Chong-Wah Ngo,et al.  Transferrable Prototypical Networks for Unsupervised Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[64]  Lei Zhang,et al.  Unsupervised Domain Adaptation Using Robust Class-Wise Matching , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[65]  Philip S. Yu,et al.  Transfer Feature Learning with Joint Distribution Adaptation , 2013, 2013 IEEE International Conference on Computer Vision.

[66]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[67]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[68]  Xingyuan Wang,et al.  Image Description With Polar Harmonic Fourier Moments , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[69]  Ke Lu,et al.  Structured Domain Adaptation , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[70]  Yi Yang,et al.  Contrastive Adaptation Network for Unsupervised Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).