Universal Multi-Source Domain Adaptation

Unsupervised domain adaptation enables intelligent models to transfer knowledge from a labeled source domain to a similar but unlabeled target domain. Recent study reveals that knowledge can be transferred from one source domain to another unknown target domain, called Universal Domain Adaptation (UDA). However, in the real-world application, there are often more than one source domain to be exploited for domain adaptation. In this paper, we formally propose a more general domain adaptation setting, universal multi-source domain adaptation (UMDA), where the label sets of multiple source domains can be different and the label set of target domain is completely unknown. The main challenges in UMDA are to identify the common label set between each source domain and target domain, and to keep the model scalable as the number of source domains increases. To address these challenges, we propose a universal multi-source adaptation network (UMAN) to solve the domain adaptation problem without increasing the complexity of the model in various UMDA settings. In UMAN, we estimate the reliability of each known class in the common label set via the prediction margin, which helps adversarial training to better align the distributions of multiple source domains and target domain in the common label set. Moreover, the theoretical guarantee for UMAN is also provided. Massive experimental results show that existing UDA and multi-source DA (MDA) methods cannot be directly applied to UMDA and the proposed UMAN achieves the state-of-the-art performance in various UMDA settings.

[1]  Michael I. Jordan,et al.  Unsupervised Domain Adaptation with Residual Transfer Networks , 2016, NIPS.

[2]  Juergen Gall,et al.  Open Set Domain Adaptation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[3]  R. Venkatesh Babu,et al.  Universal Source-Free Domain Adaptation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Haifeng Shen,et al.  Mutual Learning Network for Multi-Source Domain Adaptation , 2020, ArXiv.

[5]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[6]  Liang Lin,et al.  Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  Zi Huang,et al.  Progressive Graph Learning for Open-Set Domain Adaptation , 2020, ICML.

[8]  Michael I. Jordan,et al.  Universal Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  David J. Kriegman,et al.  Image to Image Translation for Domain Adaptation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  David E. Carlson,et al.  Extracting Relationships by Multi-Domain Matching , 2018, NeurIPS.

[11]  Yuchen Zhang,et al.  Bridging Theory and Algorithm for Domain Adaptation , 2019, ICML.

[12]  Kurt Keutzer,et al.  Multi-source Distilling Domain Adaptation , 2020, AAAI.

[13]  Mehryar Mohri,et al.  Algorithms and Theory for Multiple-Source Adaptation , 2018, NeurIPS.

[14]  Regina Barzilay,et al.  Multi-Source Domain Adaptation with Mixture of Experts , 2018, EMNLP.

[15]  Tat-Seng Chua,et al.  Multi-source Domain Adaptation for Visual Sentiment Classification , 2020, AAAI.

[16]  Hau-San Wong,et al.  Generating Target Image-Label Pairs for Unsupervised Domain Adaptation , 2020, IEEE Transactions on Image Processing.

[17]  Juergen Gall,et al.  Open Set Domain Adaptation for Image and Action Recognition , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Koby Crammer,et al.  Learning Bounds for Domain Adaptation , 2007, NIPS.

[19]  Cordelia Schmid,et al.  Label-Embedding for Image Classification , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Daniel Cremers,et al.  Associative Domain Adaptation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[21]  José M. F. Moura,et al.  Adversarial Multiple Source Domain Adaptation , 2018, NeurIPS.

[22]  V. Koltchinskii,et al.  Empirical margin distributions and bounding the generalization error of combined classifiers , 2002, math/0405343.

[23]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[24]  José M. F. Moura,et al.  Multiple Source Domain Adaptation with Adversarial Learning , 2018, ICLR.

[25]  Jie Lu,et al.  Multi-Source Domain Adaptation with Distribution Fusion and Relationship Extraction , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[26]  Deqing Wang,et al.  Aligning Domain-Specific Distribution and Classifier for Cross-Domain Classification from Multiple Sources , 2019, AAAI.

[27]  Baochang Zhang,et al.  Aligning Discriminative and Representative Features: An Unsupervised Domain Adaptation Method for Building Damage Assessment , 2020, IEEE Transactions on Image Processing.

[28]  Sheng-De Wang,et al.  Detach and Adapt: Learning Cross-Domain Disentangled Deep Representation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Michael I. Jordan,et al.  Conditional Adversarial Domain Adaptation , 2017, NeurIPS.

[30]  Jianmin Wang,et al.  Partial Adversarial Domain Adaptation , 2018, ECCV.

[31]  Shiguang Shan,et al.  Duplex Generative Adversarial Network for Unsupervised Domain Adaptation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[32]  Mehrtash Harandi,et al.  Domain Adaptation by Joint Distribution Invariant Projections , 2020, IEEE Transactions on Image Processing.

[33]  Yishay Mansour,et al.  Domain Adaptation with Multiple Sources , 2008, NIPS.

[34]  Trevor Darrell,et al.  Deep Domain Confusion: Maximizing for Domain Invariance , 2014, CVPR 2014.

[35]  Aaron Chadha,et al.  Improved Techniques for Adversarial Discriminative Domain Adaptation , 2020, IEEE Transactions on Image Processing.

[36]  Sethuraman Panchanathan,et al.  Deep Hashing Network for Unsupervised Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Donghyun Kim,et al.  Universal Domain Adaptation through Self Supervision , 2020, NeurIPS.

[38]  Trevor Darrell,et al.  Simultaneous Deep Transfer Across Domains and Tasks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[39]  Dumitru Erhan,et al.  Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Nikolaos Papanikolopoulos,et al.  Scalable Active Learning for Multiclass Image Classification , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Xian-Sheng Hua,et al.  Two-Dimensional Multilabel Active Learning with an Efficient Online Adaptation Model for Image Classification , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Jianmin Wang,et al.  Partial Transfer Learning with Selective Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[43]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Tatsuya Harada,et al.  Maximum Classifier Discrepancy for Unsupervised Domain Adaptation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[45]  Bo Wang,et al.  Moment Matching for Multi-Source Domain Adaptation , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[46]  Gabriela Csurka,et al.  Distance-Based Image Classification: Generalizing to New Classes at Near-Zero Cost , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Shang-Hong Lai,et al.  AugGAN: Cross Domain Adaptation with GAN-Based Data Augmentation , 2018, ECCV.

[48]  Kurt Keutzer,et al.  MADAN: Multi-source Adversarial Domain Aggregation Network for Domain Adaptation , 2020, International Journal of Computer Vision.

[49]  Trevor Darrell,et al.  Adapting Visual Category Models to New Domains , 2010, ECCV.

[50]  Cheng Wu,et al.  A Graph Embedding Framework for Maximum Mean Discrepancy-Based Domain Adaptation Algorithms , 2020, IEEE Transactions on Image Processing.

[51]  Jiwen Lu,et al.  Runtime Network Routing for Efficient Image Classification , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Kate Saenko,et al.  VisDA: A Synthetic-to-Real Benchmark for Visual Domain Adaptation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[54]  Silvio Savarese,et al.  Adversarial Feature Augmentation for Unsupervised Domain Adaptation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[55]  Yuan Shi,et al.  Geodesic flow kernel for unsupervised domain adaptation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Kurt Keutzer,et al.  Multi-source Domain Adaptation in the Deep Learning Era: A Systematic Survey , 2020, ArXiv.

[57]  PerronninFlorent,et al.  Good Practice in Large-Scale Learning for Image Classification , 2014 .

[58]  Mohammed Bennamoun,et al.  Deep Reconstruction Models for Image Set Classification , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[60]  Jing Zhang,et al.  Importance Weighted Adversarial Nets for Partial Domain Adaptation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[61]  Tatsuya Harada,et al.  Open Set Domain Adaptation by Backpropagation , 2018, ECCV.

[62]  Carlos D. Castillo,et al.  Generate to Adapt: Aligning Domains Using Generative Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[63]  Lior Wolf,et al.  A Sample Selection Approach for Universal Domain Adaptation , 2020, ArXiv.

[64]  Subhasis Chaudhuri,et al.  Unsupervised Multi-source Domain Adaptation Driven by Deep Adversarial Ensemble Learning , 2019, GCPR.