Integration of Global and Local Metrics for Domain Adaptation Learning Via Dimensionality Reduction

Domain adaptation learning (DAL) investigates how to perform a task across different domains. In this paper, we present a kernelized local-global approach to solve domain adaptation problems. The basic idea of the proposed method is to consider the global and local information regarding the domains (e.g., maximum mean discrepancy and intraclass distance) and to convert the domain adaptation problem into a bi-object optimization problem via the kernel method. A solution for the optimization problem will help us identify a latent space in which the distributions of the different domains will be close to each other in the global sense, and the local properties of the labeled source samples will be preserved. Therefore, classic classification algorithms can be used to recognize unlabeled target domain data, which has a significant difference on the source samples. Based on the analysis, we validate the proposed algorithm using four different sources of data: synthetic, textual, object, and facial image. The experimental results indicate that the proposed method provides a reasonable means to improve DAL algorithms.

[1]  Ivor W. Tsang,et al.  Domain Adaptation from Multiple Sources : A Domain-Dependent Regularization Approach , 2012 .

[2]  Xuelong Li,et al.  Flowing on Riemannian Manifold: Domain Adaptation by Shifting Covariance , 2014, IEEE Transactions on Cybernetics.

[3]  Lorenzo Bruzzone,et al.  Domain Adaptation Problems: A DASVM Classification Technique and a Circular Validation Strategy , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Pong C. Yuen,et al.  Domain Transfer Support Vector Ranking for Person Re-identification without Target Camera Label Information , 2013, 2013 IEEE International Conference on Computer Vision.

[5]  Qiang Yang,et al.  Transfer Learning via Dimensionality Reduction , 2008, AAAI.

[6]  Yuan Shi,et al.  Information-Theoretical Learning of Discriminative Clusters for Unsupervised Domain Adaptation , 2012, ICML.

[7]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[8]  Motoaki Kawanabe,et al.  Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation , 2007, NIPS.

[9]  Hal Daumé,et al.  Frustratingly Easy Domain Adaptation , 2007, ACL.

[10]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[11]  Trevor Darrell,et al.  Adapting Visual Category Models to New Domains , 2010, ECCV.

[12]  G. Griffin,et al.  Caltech-256 Object Category Dataset , 2007 .

[13]  Qiang Yang,et al.  Transferring Naive Bayes Classifiers for Text Classification , 2007, AAAI.

[14]  John Blitzer,et al.  Domain Adaptation with Structural Correspondence Learning , 2006, EMNLP.

[15]  Le Song,et al.  A Hilbert Space Embedding for Distributions , 2007, Discovery Science.

[16]  Zhaohong Deng,et al.  Generalized Hidden-Mapping Ridge Regression, Knowledge-Leveraged Inductive Transfer Learning for Neural Networks, Fuzzy Systems and Kernel Methods , 2014, IEEE Transactions on Cybernetics.

[17]  A. Martínez,et al.  The AR face databasae , 1998 .

[18]  Rama Chellappa,et al.  Visual Domain Adaptation: A survey of recent advances , 2015, IEEE Signal Processing Magazine.

[19]  Ivor W. Tsang,et al.  Domain Transfer Multiple Kernel Learning , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[21]  Xin Zhang,et al.  Learning Robotic Hand-eye Coordination Through a Developmental Constraint Driven Approach , 2013, Int. J. Autom. Comput..

[22]  Qiang Yang,et al.  Boosting for transfer learning , 2007, ICML '07.

[23]  Rama Chellappa,et al.  A Grassmann manifold-based domain adaptation approach , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[24]  James J. Jiang.  A Literature Survey on Domain Adaptation of Statistical Classifiers , 2007 .

[25]  Koby Crammer,et al.  Learning Bounds for Domain Adaptation , 2007, NIPS.

[26]  Ingo Steinwart,et al.  On the Influence of the Kernel on the Consistency of Support Vector Machines , 2002, J. Mach. Learn. Res..

[27]  Rama Chellappa,et al.  Domain adaptation for object recognition: An unsupervised approach , 2011, 2011 International Conference on Computer Vision.

[28]  Seiichi Ozawa,et al.  A Multitask Learning Model for Online Pattern Recognition , 2009, IEEE Transactions on Neural Networks.

[29]  Yong Wang,et al.  MOMMOP: Multiobjective Optimization for Locating Multiple Optimal Solutions of Multimodal Optimization Problems , 2015, IEEE Transactions on Cybernetics.

[30]  Ivor W. Tsang,et al.  Combating Negative Transfer From Predictive Distribution Differences , 2013, IEEE Transactions on Cybernetics.

[31]  Fan Zhang,et al.  Fuzzy neural network based dynamic path planning , 2012, 2012 International Conference on Machine Learning and Cybernetics.

[32]  Yoshua Bengio,et al.  Bias learning, knowledge sharing , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[33]  Ben Goertzel,et al.  Improving machine vision via incorporating expectation-maximization into Deep Spatio-Temporal learning , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[34]  Bianca Zadrozny,et al.  Learning and evaluating classifiers under sample selection bias , 2004, ICML.

[35]  Yuan Shi,et al.  Geodesic flow kernel for unsupervised domain adaptation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Daumé,et al.  Frustratingly Easy Semi-Supervised Domain Adaptation , 2010 .

[37]  Tong Zhang,et al.  A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , 2005, J. Mach. Learn. Res..

[38]  Tinne Tuytelaars,et al.  Unsupervised Visual Domain Adaptation Using Subspace Alignment , 2013, 2013 IEEE International Conference on Computer Vision.

[39]  Koby Crammer,et al.  Analysis of Representations for Domain Adaptation , 2006, NIPS.

[40]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[41]  Raghuraman Gopalan,et al.  Model-Driven Domain Adaptation on Product Manifolds for Unconstrained Face Recognition , 2014, International Journal of Computer Vision.

[42]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[43]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[44]  Ruth Urner,et al.  Domain adaptation–can quantity compensate for quality? , 2013, Annals of Mathematics and Artificial Intelligence.

[45]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[46]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[47]  Korris Fu-Lai Chung,et al.  On minimum distribution discrepancy support vector machine for domain adaptation , 2012, Pattern Recognit..

[48]  Bernhard Schölkopf,et al.  A Kernel Method for the Two-Sample-Problem , 2006, NIPS.