Sub-domain adaptation learning methodology

Regarded as global methods, Maximum Mean Discrepancy (MMD) based transfer learning frameworks only reflect the global distribution discrepancy and structural differences between domains; they can reflect neither the inner local distribution discrepancy nor the structural differences between domains. To address this problem, a novel transfer learning framework with local learning ability, a Sub-domain Adaptation Learning Framework (SDAL), is proposed. In this framework, a Projected Maximum Local Weighted Mean Discrepancy (PMLMD) is constructed by integrating the theory and method of Local Weighted Mean (LWM) into MMD. PMLMD reflects global distribution discrepancy between domains through accumulating local distribution discrepancies between the local sub-domains in domains. In particular, we formulate in theory that PMLMD is one of the generalized measures of MMD. On the basis of SDAL, two novel methods are proposed by using Multi-label Classifiers (MLC) and Support Vector Machine (SVM). Finally, tests on artificial data sets, high dimensional text data sets and face data sets show the SDAL-based transfer learning methods are superior to or at least comparable with benchmarking methods.

[1]  Ivor W. Tsang,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 1 Domain Adaptation from Multiple Sources: A Domain- , 2022 .

[2]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[3]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[4]  Ivor W. Tsang,et al.  Extracting discriminative concepts for domain adaptation in text mining , 2009, KDD.

[5]  Zhi-Hua Zhou,et al.  New Semi-Supervised Classification Method Based on Modified Cluster Assumption , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Jun Huan,et al.  Large margin transductive transfer learning , 2009, CIKM.

[7]  Philip S. Yu,et al.  Adaptation Regularization: A General Framework for Transfer Learning , 2014, IEEE Transactions on Knowledge and Data Engineering.

[8]  Jennifer G. Dy,et al.  Fast semi-supervised SVM classifiers using a priori metric information , 2008, Optim. Methods Softw..

[9]  Simon Coupland,et al.  Fuzzy Transfer Learning: Methodology and application , 2015, Inf. Sci..

[10]  Longbing Cao,et al.  Knowledge-leverage based TSK fuzzy system with improved knowledge transfer , 2014, 2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[11]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[12]  Songcan Chen,et al.  New Least Squares Support Vector Machines Based on Matrix Patterns , 2007, Neural Processing Letters.

[13]  Peter Harremoës,et al.  Rényi Divergence and Kullback-Leibler Divergence , 2012, IEEE Transactions on Information Theory.

[14]  Jieping Ye,et al.  Extracting shared subspace for multi-label classification , 2008, KDD.

[15]  Andrew W. Moore,et al.  Locally Weighted Learning , 1997, Artificial Intelligence Review.

[16]  Deli Zhao,et al.  Graph Degree Linkage: Agglomerative Clustering on a Directed Graph , 2012, ECCV.

[17]  Qiang Yang,et al.  Spectral domain-transfer learning , 2008, KDD.

[18]  Shitong Wang,et al.  Sparsity regularization label propagation for domain adaptation learning , 2014, Neurocomputing.

[19]  Cecilio Angulo,et al.  A Note on the Bias in SVMs for Multiclassification , 2008, IEEE Transactions on Neural Networks.

[20]  Seiichi Ozawa,et al.  A Multitask Learning Model for Online Pattern Recognition , 2009, IEEE Transactions on Neural Networks.

[21]  Jiawei Han,et al.  Knowledge transfer via multiple model local structure mapping , 2008, KDD.

[22]  Barbara Caputo,et al.  Learning Categories From Few Examples With Multi Model Knowledge Transfer , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Shigeo Abe,et al.  Fuzzy least squares support vector machines for multiclass problems , 2003, Neural Networks.

[24]  Ting Wang,et al.  Kernel Sparse Representation-Based Classifier , 2012, IEEE Transactions on Signal Processing.

[25]  Hans-Peter Kriegel,et al.  Integrating structured biological data by Kernel Maximum Mean Discrepancy , 2006, ISMB.

[26]  Ivor W. Tsang,et al.  Transfer Ordinal Label Learning , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[27]  John M. Lee.  Riemannian Manifolds: An Introduction to Curvature , 1997 .

[28]  Jun Huan,et al.  Knowledge Transfer with Low-Quality Data: A Feature Extraction Issue , 2012, IEEE Trans. Knowl. Data Eng..

[29]  Ivor W. Tsang,et al.  Domain Transfer Multiple Kernel Learning , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[31]  Eric Eaton,et al.  An automated measure of MDP similarity for transfer in reinforcement learning , 2014, AAAI 2014.

[32]  Takafumi Kanamori,et al.  A Least-squares Approach to Direct Importance Estimation , 2009, J. Mach. Learn. Res..

[33]  Korris Fu-Lai Chung,et al.  On minimum distribution discrepancy support vector machine for domain adaptation , 2012, Pattern Recognit..

[34]  Bernhard Schölkopf,et al.  Correcting Sample Selection Bias by Unlabeled Data , 2006, NIPS.

[35]  Lorenzo Bruzzone,et al.  Domain Adaptation Problems: A DASVM Classification Technique and a Circular Validation Strategy , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Deli Zhao,et al.  Linear Laplacian Discrimination for Feature Extraction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Qiang Yang,et al.  Transfer Learning via Dimensionality Reduction , 2008, AAAI.

[38]  Rama Chellappa,et al.  Unsupervised Adaptation Across Domain Shifts by Generating Intermediate Data Representations , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Dan Zhang,et al.  Multi-view transfer learning with a large margin approach , 2011, KDD.