Class conditional distribution alignment for domain adaptation

In this paper, we study the problem of domain adaptation, which is a crucial ingredient in transfer learning with two domains, that is, the source domain with labeled data and the target domain with none or few labels. Domain adaptation aims to extract knowledge from the source domain to improve the performance of the learning task in the target domain. A popular approach to handle this problem is via adversarial training, which is explained by the H Δ H -distance theory. However, traditional adversarial network architectures just align the marginal feature distribution in the feature space. The alignment of class condition distribution is not guaranteed. Therefore, we proposed a novel method based on pseudo labels and the cluster assumption to avoid the incorrect class alignment in the feature space. The experiments demonstrate that our framework improves the accuracy on typical transfer learning tasks.

[1]  Shih-Fu Chang,et al.  Deep Transfer Network: Unsupervised Domain Adaptation , 2015, ArXiv.

[2]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[3]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Mei Wang,et al.  Deep Visual Domain Adaptation: A Survey , 2018, Neurocomputing.

[5]  Jiwen Lu,et al.  Deep transfer metric learning , 2015, CVPR.

[6]  George Trigeorgis,et al.  Domain Separation Networks , 2016, NIPS.

[7]  Jianmin Wang,et al.  Partial Transfer Learning with Selective Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  Mengjie Zhang,et al.  Deep Reconstruction-Classification Networks for Unsupervised Domain Adaptation , 2016, ECCV.

[9]  Francesco Borrelli,et al.  Learning Model Predictive Control for Iterative Tasks. A Data-Driven Control Framework , 2016, IEEE Transactions on Automatic Control.

[10]  Shin Ishii,et al.  Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Trevor Darrell,et al.  Deep Domain Confusion: Maximizing for Domain Invariance , 2014, CVPR 2014.

[12]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[13]  Thomas Hofmann,et al.  Analysis of Representations for Domain Adaptation , 2007 .

[14]  Chao Yang,et al.  A Survey on Deep Transfer Learning , 2018, ICANN.

[15]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[16]  Yang Ming.  Variational Bayesian data analysis on manifold , 2018, Control Theory and Technology.

[17]  Jaehong Park,et al.  Analyzing OTDR Measurement Data Using the Kalman Filter , 2008, IEEE Transactions on Instrumentation and Measurement.

[18]  Stefano Ermon,et al.  A DIRT-T Approach to Unsupervised Domain Adaptation , 2018, ICLR.

[19]  Namil Kim,et al.  Pixel-Level Domain Transfer , 2016, ECCV.

[20]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[21]  Dumitru Erhan,et al.  Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Alexander Zien,et al.  Semi-Supervised Classification by Low Density Separation , 2005, AISTATS.

[23]  Tatsuya Harada,et al.  Maximum Classifier Discrepancy for Unsupervised Domain Adaptation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Ming-Yu Liu,et al.  Coupled Generative Adversarial Networks , 2016, NIPS.

[25]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[26]  Håkan Hjalmarsson,et al.  Iterative Data-Driven ${\cal H}_{\infty}$ Norm Estimation of Multivariable Systems With Application to Robust Active Vibration Isolation , 2014, IEEE Transactions on Control Systems Technology.

[27]  Tatsuya Harada,et al.  Asymmetric Tri-training for Unsupervised Domain Adaptation , 2017, ICML.

[28]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[29]  Rogério Schmidt Feris,et al.  Co-regularized Alignment for Unsupervised Domain Adaptation , 2018, NeurIPS.

[30]  Tao Zhang,et al.  Deep Model Based Domain Adaptation for Fault Diagnosis , 2017, IEEE Transactions on Industrial Electronics.

[31]  Silvio Savarese,et al.  Adversarial Feature Augmentation for Unsupervised Domain Adaptation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.