Transfer sparse machine: matching joint distribution by subspace learning and classifier transduction

Transfer learning problem aims at matching the joint distributions of the source and the target datasets so that the model learned from the source dataset can be applied to the target dataset. Unfortunately, the joint distribution of the database may be very hard to estimate in many applications. Since the joint distribution can be written as the product of the marginal and the conditional distributions, we propose the TSM, which tries to match the latter two distributions respectively, instead of directly matching the joint distributions. The proposed TSM consists of two parts: a feature learning part which matches the marginal distributions by learning a shared feature space, and a classifier training part which matches the conditional distributions by training an adaptive classifier in the shared feature space. Comprehensive experiments prove that the superior performance of the TSM on several transfer learning datasets. And the improvements are 12.86% on the USPS/MNIST dataset and 9.01% on the PIE1/PIE2 dataset compared to the best baseline.

[1]  Philip S. Yu,et al.  Transfer Sparse Coding for Robust Image Representation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Philip S. Yu,et al.  Transfer Joint Matching for Unsupervised Domain Adaptation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[4]  Philip S. Yu,et al.  Adaptation Regularization: A General Framework for Transfer Learning , 2014, IEEE Transactions on Knowledge and Data Engineering.

[5]  Yan Liu,et al.  Learning with Minimum Supervision: A General Framework for Transductive Transfer Learning , 2011, 2011 IEEE 11th International Conference on Data Mining.

[6]  Jun Huan,et al.  Knowledge Transfer with Low-Quality Data: A Feature Extraction Issue , 2011, IEEE Transactions on Knowledge and Data Engineering.

[7]  Michael Elad,et al.  Probabilistic Subspace Clustering Via Sparse Representations , 2013, IEEE Signal Processing Letters.

[8]  Ivor W. Tsang,et al.  Extracting discriminative concepts for domain adaptation in text mining , 2009, KDD.

[9]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[10]  Mikhail Belkin,et al.  Laplacian Support Vector Machines Trained in the Primal , 2009, J. Mach. Learn. Res..

[11]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[12]  Lorenzo Bruzzone,et al.  Domain Adaptation Problems: A DASVM Classification Technique and a Circular Validation Strategy , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Chun Chen,et al.  Graph Regularized Sparse Coding for Image Representation , 2011, IEEE Transactions on Image Processing.

[14]  Qingming Huang,et al.  Transferring Boosted Detectors Towards Viewpoint and Scene Adaptiveness , 2011, IEEE Transactions on Image Processing.

[15]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.