Transferable Feature Selection for Unsupervised Domain Adaptation

Domain adaptation aims at extracting knowledge from auxiliary source domains to assist the learning task in a target domain. In classification problems, since the distributions of the source and target domains are different, directly using source data to build a classifier for the target domain may hamper the classification performance on the target data. Fortunately, in many tasks, there can be some features that are transferable, i.e., the source and target domains share similar properties. On the other hand, it is common that the source data contain noisy features which may degrade the learning performance in the target domain. This issue, however, is barely studied in existing works. In this paper, we propose to find a feature subset that is transferable across the source and target domains. As a result, the domain discrepancy measured on the selected features can be reduced. Moreover, we seek to find the most discriminative features for classification. To achieve the above goals, we formulate a new sparse learning model that is able to jointly reduce the domain discrepancy and select informative features for classification. We develop two optimization algorithms to address the derived learning problem. Extensive experiments on real-world data sets demonstrate the effectiveness of the proposed method.

[1]  Qingyao Wu,et al.  Iterative Refinement for Multi-Source Visual Domain Adaptation , 2022, IEEE Transactions on Knowledge and Data Engineering.

[2]  M. Ng,et al.  Domain-attention Conditional Wasserstein Distance for Multi-source Domain Adaptation , 2020, ACM Trans. Intell. Syst. Technol..

[3]  Qingyao Wu,et al.  Geometric Knowledge Embedding for unsupervised domain adaptation , 2020, Knowl. Based Syst..

[4]  Feiping Nie,et al.  Detecting Coherent Groups in Crowd Scenes by Multiview Clustering , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Yue Cao,et al.  Transferable Representation Learning with Deep Adaptation Networks , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Namil Kim,et al.  Drop to Adapt: Learning Discriminative Features for Unsupervised Domain Adaptation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[7]  Hanrui Wu,et al.  Informative Feature Selection for Domain Adaptation , 2019, IEEE Access.

[8]  Qingyao Wu,et al.  Learning Sparse PCA with Stabilized ADMM Method on Stiefel Manifold , 2019, IEEE Transactions on Knowledge and Data Engineering.

[9]  Xuelong Li,et al.  Weakly Supervised Adversarial Domain Adaptation for Semantic Segmentation in Urban Scenes , 2019, IEEE Transactions on Image Processing.

[10]  Amaury Lendasse,et al.  Domain Adaption via Feature Selection on Explicit Feature Map , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Jian Yang,et al.  Manifold Criterion Guided Transfer Learning via Intermediate Domain Generation , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification Using Rearranged Local Features , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Ming Shao,et al.  Graph Adaptive Knowledge Transfer for Unsupervised Domain Adaptation , 2018, ECCV.

[14]  Wen Li,et al.  Semi-Supervised Optimal Transport for Heterogeneous Domain Adaptation , 2018, IJCAI.

[15]  Carole Lartizien,et al.  Feature Selection for Unsupervised Domain Adaptation using Optimal Transport , 2018, ECML/PKDD.

[16]  Zhiguo Cao,et al.  An Embarrassingly Simple Approach to Visual Domain Adaptation , 2018, IEEE Transactions on Image Processing.

[17]  Daniel Cremers,et al.  Associative Domain Adaptation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[18]  Michael K. Ng,et al.  Learning Discriminative Correlation Subspace for Heterogeneous Domain Adaptation , 2017, IJCAI.

[19]  Zhiguo Cao,et al.  When Unsupervised Domain Adaptation Meets Tensor Representations , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Mingda Zhang,et al.  Automatic Understanding of Image and Video Advertisements , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Qingyao Wu,et al.  Online Transfer Learning with Multiple Homogeneous or Heterogeneous Sources , 2017, IEEE Transactions on Knowledge and Data Engineering.

[22]  Hui Xiong,et al.  A Unified Framework for Metric Transfer Learning , 2017, IEEE Transactions on Knowledge and Data Engineering.

[23]  Nicolas Courty,et al.  Joint distribution optimal transportation for domain adaptation , 2017, NIPS.

[24]  Jing Zhang,et al.  Joint Geometrical and Statistical Alignment for Visual Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Qilong Wang,et al.  Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Yiran Chen,et al.  A Compact DNN: Approaching GoogLeNet-Level Accuracy of Classification and Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Fernando De la Torre,et al.  Selective Transfer Machine for Personalized Facial Expression Analysis , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Philip S. Yu,et al.  Deep Learning of Transferable Representation for Scalable Domain Adaptation , 2016, IEEE Transactions on Knowledge and Data Engineering.

[30]  Ke Lu,et al.  Joint Feature Selection and Structure Preservation for Domain Adaptation , 2016, IJCAI.

[31]  Yu-Chiang Frank Wang,et al.  Learning Cross-Domain Landmarks for Heterogeneous Domain Adaptation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Michael I. Jordan,et al.  Deep Transfer Learning with Joint Adaptation Networks , 2016, ICML.

[33]  Michael I. Jordan,et al.  Unsupervised Domain Adaptation with Residual Transfer Networks , 2016, NIPS.

[34]  Ivor W. Tsang,et al.  Transfer Learning for Cross-Language Text Categorization through Active Correspondences Construction , 2016, AAAI.

[35]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Xiaoqun Zhang,et al.  A Note on the Complexity of Proximal Iterative Hard Thresholding Algorithm , 2015 .

[37]  Kate Saenko,et al.  Return of Frustratingly Easy Domain Adaptation , 2015, AAAI.

[38]  Philip S. Yu,et al.  Domain Invariant Transfer Kernel Learning , 2015, IEEE Transactions on Knowledge and Data Engineering.

[39]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[40]  Ling Shao,et al.  Transfer Learning for Visual Categorization: A Survey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[41]  Rama Chellappa,et al.  Visual Domain Adaptation: A survey of recent advances , 2015, IEEE Signal Processing Magazine.

[42]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[43]  Nicolas Courty,et al.  Optimal Transport for Domain Adaptation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Bin Li,et al.  Online Transfer Learning , 2014, Artif. Intell..

[45]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[46]  Nicolas Courty,et al.  Domain Adaptation with Regularized Optimal Transport , 2014, ECML/PKDD.

[47]  Philip S. Yu,et al.  Transfer Joint Matching for Unsupervised Domain Adaptation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Ivor W. Tsang,et al.  Hybrid Heterogeneous Transfer Learning through Deep Learning , 2014, AAAI.

[49]  Ivor W. Tsang,et al.  Learning With Augmented Features for Supervised and Semi-Supervised Heterogeneous Domain Adaptation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Philip S. Yu,et al.  Adaptation Regularization: A General Framework for Transfer Learning , 2014, IEEE Transactions on Knowledge and Data Engineering.

[51]  Philip S. Yu,et al.  Transfer Feature Learning with Joint Distribution Adaptation , 2013, 2013 IEEE International Conference on Computer Vision.

[52]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[53]  Weixiong Zhang,et al.  Domain Adaptation with Topical Correspondence Learning , 2013, IJCAI.

[54]  Philip S. Yu,et al.  Transfer Sparse Coding for Robust Image Representation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Fernando De la Torre,et al.  Selective Transfer Machine for Personalized Facial Action Unit Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Ivor W. Tsang,et al.  Minimax Sparse Logistic Regression for Very High-Dimensional Feature Selection , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[57]  Ivor W. Tsang,et al.  Towards ultrahigh dimensional feature selection for big data , 2012, J. Mach. Learn. Res..

[58]  Jianmin Wang,et al.  Transfer Learning with Graph Co-Regularization , 2012, IEEE Transactions on Knowledge and Data Engineering.

[59]  Yuan Shi,et al.  Information-Theoretical Learning of Discriminative Clusters for Unsupervised Domain Adaptation , 2012, ICML.

[60]  Ivor W. Tsang,et al.  Learning with Augmented Features for Heterogeneous Domain Adaptation , 2012, ICML.

[61]  Kilian Q. Weinberger,et al.  Marginalized Denoising Autoencoders for Domain Adaptation , 2012, ICML.

[62]  Yuan Shi,et al.  Geodesic flow kernel for unsupervised domain adaptation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  John Blitzer,et al.  Co-Training for Domain Adaptation , 2011, NIPS.

[64]  Jaime G. Carbonell,et al.  Feature Selection for Transfer Learning , 2011, ECML/PKDD.

[65]  Eric Eaton,et al.  Selective Transfer Between Learning Tasks Using Task-Based Boosting , 2011, AAAI.

[66]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[67]  Trevor Darrell,et al.  Adapting Visual Category Models to New Domains , 2010, ECCV.

[68]  Ivor W. Tsang,et al.  Learning Sparse SVM for Feature Selection on Very High Dimensional Datasets , 2010, ICML.

[69]  Ivor W. Tsang,et al.  Visual Event Recognition in Videos by Learning from Web Data , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[70]  Lorenzo Bruzzone,et al.  Domain Adaptation Problems: A DASVM Classification Technique and a Circular Validation Strategy , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[71]  Massih-Reza Amini,et al.  Learning from Multiple Partially Observed Views - an Application to Multilingual Text Categorization , 2009, NIPS.

[72]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[73]  C. Villani Optimal Transport: Old and New , 2008 .

[74]  Chih-Jen Lin,et al.  A dual coordinate descent method for large-scale linear SVM , 2008, ICML '08.

[75]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[76]  Qiang Yang,et al.  Adaptive Localization in a Dynamic WiFi Environment through Multi-view Learning , 2007, AAAI.

[77]  Qiang Yang,et al.  Boosting for transfer learning , 2007, ICML '07.

[78]  John Blitzer,et al.  Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification , 2007, ACL.

[79]  Bernhard Schölkopf,et al.  Correcting Sample Selection Bias by Unlabeled Data , 2006, NIPS.

[80]  Bernhard Schölkopf,et al.  A Kernel Method for the Two-Sample-Problem , 2006, NIPS.

[81]  Hans-Peter Kriegel,et al.  Integrating structured biological data by Kernel Maximum Mean Discrepancy , 2006, ISMB.

[82]  L. Kantorovich On the Translocation of Masses , 2006 .

[83]  Roland Glowinski,et al.  On Alternating Direction Methods of Multipliers: A Historical Perspective , 2014, Modeling, Simulation and Optimization for Science and Technology.