Feature Selection for Unsupervised Domain Adaptation using Optimal Transport

In this paper, we propose a new feature selection method for unsuper-vised domain adaptation based on the emerging optimal transportation theory. We build upon a recent theoretical analysis of optimal transport in domain adaptation and show that it can directly suggest a feature selection procedure leveraging the shift between the domains. Based on this, we propose a novel algorithm that aims to sort features by their similarity across the source and target domains, where the order is obtained by analyzing the coupling matrix representing the solution of the proposed optimal transportation problem. We evaluate our method on a well-known benchmark data set and illustrate its capability of selecting correlated features leading to better classification performances. Furthermore, we show that the proposed algorithm can be used as a pre-processing step for existing domain adaptation techniques ensuring an important speed-up in terms of the computational time while maintaining comparable results. Finally, we validate our algorithm on clinical imaging databases for computer-aided diagnosis task with promising results.

[1]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[3]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[4]  M. Talagrand Concentration of measure and isoperimetric inequalities in product spaces , 1994, math/9406212.

[5]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[6]  Ke Lu,et al.  Joint Feature Selection and Structure Preservation for Domain Adaptation , 2016, IJCAI.

[7]  Trevor Darrell,et al.  Adapting Visual Category Models to New Domains , 2010, ECCV.

[8]  Carole Lartizien,et al.  Transfer Learning for Prostate Cancer Mapping Based on Multicentric MR Imaging Databases , 2015, MLMMI@ICML.

[9]  Jaime G. Carbonell,et al.  Feature Selection for Transfer Learning , 2011, ECML/PKDD.

[10]  Philip A. Knight,et al.  The Sinkhorn-Knopp Algorithm: Convergence and Applications , 2008, SIAM J. Matrix Anal. Appl..

[11]  Wei Zhang,et al.  Cross-Subject EEG Feature Selection for Emotion Recognition Using Transfer Recursive Feature Elimination , 2017, Front. Neurorobot..

[12]  Kate Saenko,et al.  Return of Frustratingly Easy Domain Adaptation , 2015, AAAI.

[13]  C. Villani Optimal Transport: Old and New , 2008 .

[14]  Carole Lartizien,et al.  Computer-aided diagnosis of prostate cancer in the peripheral zone using multiparametric MRI , 2012, Physics in medicine and biology.

[15]  Nicolas Courty,et al.  Domain Adaptation with Regularized Optimal Transport , 2014, ECML/PKDD.

[16]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[17]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[18]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[19]  Lorenzo Bruzzone,et al.  Kernel-Based Domain-Invariant Feature Selection in Hyperspectral Images for Transfer Learning , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Rama Chellappa,et al.  Domain adaptation for object recognition: An unsupervised approach , 2011, 2011 International Conference on Computer Vision.

[21]  Tinne Tuytelaars,et al.  Unsupervised Visual Domain Adaptation Using Subspace Alignment , 2013, 2013 IEEE International Conference on Computer Vision.

[22]  Koby Crammer,et al.  Analysis of Representations for Domain Adaptation , 2006, NIPS.

[23]  Ievgen Redko,et al.  Theoretical Analysis of Domain Adaptation with Optimal Transport , 2016, ECML/PKDD.

[24]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[25]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.