Unsupervised Domain Adaptation Through Transferring both the Source-Knowledge and Target-Relatedness Simultaneously

Unsupervised domain adaptation (UDA) is an emerging research topic in the field of machine learning and pattern recognition, which aims to help the learning of unlabeled target domain by transferring knowledge from the source domain. To perform UDA, a variety of methods have been proposed, most of which concentrate on the scenario of single source and single target domain (1S1T). However, in real applications, usually single source domain with multiple target domains are involved (1SmT), which cannot be handled directly by those 1S1T models. Unfortunately, although a few related works on 1SmT UDA have been proposed, nearly none of them model the source domain knowledge and leverage the target-relatedness jointly. To overcome these shortcomings, we herein propose a more general 1SmT UDA model through transferring both the Source-Knowledge and Target-Relatedness, UDA-SKTR for short. In this way, not only the supervision knowledge from the source domain, but also the potential relatedness among the target domains are simultaneously modeled for exploitation in the process of 1SmT UDA. In addition, we construct an alternating optimization algorithm to solve the variables of the proposed model with convergence guarantee. Finally, through extensive experiments on both benchmark and real datasets, we validate the effectiveness and superiority of the proposed method.

[1]  Zhi-Hua Zhou,et al.  Automatic Age Estimation Based on Facial Aging Patterns , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Mei Wang,et al.  Deep Visual Domain Adaptation: A Survey , 2018, Neurocomputing.

[3]  Jianmin Wang,et al.  Multi-Adversarial Domain Adaptation , 2018, AAAI.

[4]  Thomas Serre,et al.  Object recognition with features inspired by visual cortex , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Min Xiao,et al.  Feature Space Independent Semi-Supervised Domain Adaptation via Kernel Matching , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Xuelong Li,et al.  Discriminative Transfer Subspace Learning via Low-Rank and Sparse Representation , 2016, IEEE Transactions on Image Processing.

[7]  Karl Ricanek,et al.  MORPH: a longitudinal image database of normal adult age-progression , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[8]  Jian Yang,et al.  Manifold Criterion Guided Transfer Learning via Intermediate Domain Generation , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Lemao Liu,et al.  Instance Weighting for Neural Machine Translation Domain Adaptation , 2017, EMNLP.

[10]  Huanhuan Yu,et al.  Multi-target Unsupervised Domain Adaptation without Exactly Shared Categories , 2018, ArXiv.

[11]  Mehrtash Tafazzoli Harandi,et al.  Learning an Invariant Hilbert Space for Domain Adaptation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Korris Fu-Lai Chung,et al.  Transfer Spectral Clustering , 2012, ECML/PKDD.

[13]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[14]  Tinghuai Ma,et al.  Ordinal space projection learning via neighbor classes representation , 2018, Comput. Vis. Image Underst..

[15]  Chu-Song Chen,et al.  Cross-Age Reference Coding for Age-Invariant Face Recognition and Retrieval , 2014, ECCV.

[16]  Qiang Yang,et al.  Self-taught clustering , 2008, ICML '08.

[17]  George Loizou,et al.  Computer vision and pattern recognition , 2007, Int. J. Comput. Math..

[18]  Saeid Nahavandi,et al.  Extreme learning machine based transfer learning algorithms: A survey , 2017, Neurocomputing.

[19]  Gao Huang,et al.  Prediction Reweighting for Domain Adaptation , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Dong Xu,et al.  Recent Advances in Transfer Learning for Cross-Dataset Visual Recognition , 2017, ACM Comput. Surv..

[21]  Lei Zhang,et al.  Guided Learning: A New Paradigm for Multi-task Classification , 2018, CCBR.

[22]  Stefanos Zafeiriou,et al.  AgeDB: The First Manually Collected, In-the-Wild Age Database , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[23]  Ivor W. Tsang,et al.  Learning with Augmented Features for Heterogeneous Domain Adaptation , 2012, ICML.

[24]  Zhi-Hua Zhou,et al.  A brief introduction to weakly supervised learning , 2018 .

[25]  Songcan Chen,et al.  Soft large margin clustering , 2013, Inf. Sci..

[26]  Bianca Zadrozny,et al.  Learning and evaluating classifiers under sample selection bias , 2004, ICML.

[27]  Yuan Shi,et al.  Geodesic flow kernel for unsupervised domain adaptation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Zhi-Hua Zhou,et al.  New Class Adaptation Via Instance Generation in One-Pass Class Incremental Learning , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[29]  Fugee Tsung,et al.  A Statistical Transfer Learning Perspective for Modeling Shape Deviations in Additive Manufacturing , 2017, IEEE Robotics and Automation Letters.

[30]  James J. Jiang.  A Literature Survey on Domain Adaptation of Statistical Classifiers , 2007 .

[31]  Motoaki Kawanabe,et al.  Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation , 2007, NIPS.

[32]  Ming Shao,et al.  Robust Multi-view Representation: A Unified Perspective from Multi-view Learning to Domain Adaption , 2018, IJCAI.

[33]  Zhaohong Deng,et al.  Transfer Prototype-Based Fuzzy Clustering , 2014, IEEE Transactions on Fuzzy Systems.

[34]  Chang Wang,et al.  Heterogeneous Domain Adaptation Using Manifold Alignment , 2011, IJCAI.

[35]  Mirkin Boris,et al.  Clustering: A Data Recovery Approach , 2012 .

[36]  ChengXiang Zhai,et al.  Instance Weighting for Domain Adaptation in NLP , 2007, ACL.

[37]  Shiliang Sun,et al.  Transfer Learning with Part-Based Ensembles , 2013, MCS.

[38]  Bernhard Schölkopf,et al.  Correcting Sample Selection Bias by Unlabeled Data , 2006, NIPS.

[39]  Wai Keung Wong,et al.  Approximate Low-Rank Projection Learning for Feature Extraction , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[40]  John Blitzer,et al.  Domain Adaptation with Structural Correspondence Learning , 2006, EMNLP.