A Study Of Alignment Mechanisms In Adversarial Domain Adaptation

Adversarial approaches (e.g. DANN [1]) are currently considered to be the most promising avenue for unsupervised domain adaptation. They aim at building a common representation space between the domains, to both i) align the source and target domains, and, ii) allow for good class discrimination in this common space. We show in this paper that this mapping to a common space can be done in different ways, and propose 5 different implementations whose performance are evaluated and compared. To this end, we have designed novel datasets/problems allowing us to make a critical analysis of the mappings and to draw important conclusions. These experiments have highlighted a second phenomenon, also little studied in the literature, which has nevertheless a major influence on the alignment performance: the inability to adapt when informative features for target are not already extracted through supervision on source. The paper provides a thorough analysis of this phenomenon.

[1]  Kate Saenko,et al.  Deep CORAL: Correlation Alignment for Deep Domain Adaptation , 2016, ECCV Workshops.

[2]  Yuichi Yoshida,et al.  Spectral Normalization for Generative Adversarial Networks , 2018, ICLR.

[3]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Taesung Park,et al.  CyCADA: Cycle-Consistent Adversarial Domain Adaptation , 2017, ICML.

[5]  Bo Wang,et al.  Moment Matching for Multi-Source Domain Adaptation , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[6]  Yuchen Zhang,et al.  Bridging Theory and Algorithm for Domain Adaptation , 2019, ICML.

[7]  François Laviolette,et al.  Domain-Adversarial Neural Networks , 2014, ArXiv.

[8]  Dumitru Erhan,et al.  Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Sethuraman Panchanathan,et al.  Deep Hashing Network for Unsupervised Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[11]  Trevor Darrell,et al.  Adapting Visual Category Models to New Domains , 2010, ECCV.

[12]  Wouter M. Kouw.  An introduction to domain adaptation and transfer learning , 2018, ArXiv.