Multi-Source Domain Adaptation with Distribution Fusion and Relationship Extraction

Transfer learning is gaining increasing attention due to its ability to leverage previously acquired knowledge to assist in completing a prediction task in a similar domain. While many existing transfer learning methods deal with single source and single target problem without considering the fact that a target domain maybe similar to multiple source domains, this work proposes a multi-source domain adaptation method based on a deep neural network. Our method contains common feature extraction, specific predictor learning and target predictor estimation. Common feature extraction explores the relationship between source domains and target domain by distribution fusion and guarantees the strength of similar source domains during training, something which has not been well considered in existing works. Specific predictor learning trains source tasks with cross-domain distribution constraint and cross-domain predictor constraint to enhance the performance of single source. Target predictor estimation employs relationship extraction and selective strategy to improve the performance of the target task and to avoid negative transfer. Experiments on real-world visual datasets show the performance of the proposed method is superior to other deep learning baselines.

[1]  Ming Shao,et al.  Incomplete Multisource Transfer Learning , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[2]  José M. F. Moura,et al.  Adversarial Multiple Source Domain Adaptation , 2018, NeurIPS.

[3]  Qiang Yang,et al.  Transitive Transfer Learning , 2015, KDD.

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Deqing Wang,et al.  Aligning Domain-Specific Distribution and Classifier for Cross-Domain Classification from Multiple Sources , 2019, AAAI.

[6]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[7]  Regina Barzilay,et al.  Multi-Source Domain Adaptation with Mixture of Experts , 2018, EMNLP.

[8]  Michael I. Jordan,et al.  Deep Transfer Learning with Joint Adaptation Networks , 2016, ICML.

[9]  Qiang Yang,et al.  Instilling Social to Physical: Co-Regularized Heterogeneous Transfer Learning , 2016, AAAI.

[10]  Yishay Mansour,et al.  Domain Adaptation with Multiple Sources , 2008, NIPS.

[11]  Jing Zhang,et al.  Joint Geometrical and Statistical Alignment for Visual Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Russell Greiner,et al.  Domain Aggregation Networks for Multi-Source Domain Adaptation , 2019, ICML.

[13]  Witold Pedrycz,et al.  Granular Fuzzy Regression Domain Adaptation in Takagi–Sugeno Fuzzy Models , 2018, IEEE Transactions on Fuzzy Systems.

[14]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[15]  Kate Saenko,et al.  Deep CORAL: Correlation Alignment for Deep Domain Adaptation , 2016, ECCV Workshops.

[16]  Nicolas Courty,et al.  Optimal Transport for Multi-source Domain Adaptation under Target Shift , 2018, AISTATS.

[17]  Liang Lin,et al.  Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Wouter M. Kouw.  An introduction to domain adaptation and transfer learning , 2018, ArXiv.

[19]  Witold Pedrycz,et al.  Fuzzy Rule-Based Domain Adaptation in Homogeneous and Heterogeneous Spaces , 2019, IEEE Transactions on Fuzzy Systems.

[20]  Philip S. Yu,et al.  Transfer Feature Learning with Joint Distribution Adaptation , 2013, 2013 IEEE International Conference on Computer Vision.

[21]  David E. Carlson,et al.  Extracting Relationships by Multi-Domain Matching , 2018, NeurIPS.

[22]  Edwin Lughofer,et al.  Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning , 2017, ICLR.

[23]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[24]  Michael I. Jordan,et al.  Towards Understanding the Transferability of Deep Representations , 2019, ArXiv.

[25]  Jinwoo Shin,et al.  Learning What and Where to Transfer , 2019, ICML.

[26]  Lei Zhang,et al.  Transfer Adaptation Learning: A Decade Survey , 2019, IEEE transactions on neural networks and learning systems.

[27]  Peng Hao,et al.  Transfer learning using computational intelligence: A survey , 2015, Knowl. Based Syst..

[28]  Koby Crammer,et al.  Analysis of Representations for Domain Adaptation , 2006, NIPS.

[29]  Tatsuya Harada,et al.  Asymmetric Tri-training for Unsupervised Domain Adaptation , 2017, ICML.

[30]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.