Transferable attention networks for adversarial domain adaptation

Abstract Domain adaptation is one of the fundamental challenges in transfer learning. How to effectively transfer knowledge from labeled source domain to unlabeled target domain is critical for domain adaptation, as it benefits to reduce the considerable performance gap due to domain shift. Existing methods of domain adaptation address this issue via matching the global features across domains. However, not all features are transferable for domain adaptation, while forcefully matching the untransferable features may lead to negative transfer. In this paper, we propose a novel method dubbed transferable attention networks (TAN) to address this issue. The proposed TAN focuses on the feature alignment by utilizing adversarial optimization. Specifically, we utilize the self-attention mechanism to weight extracted features, such that the influence of untransferable features can be effectively eliminated. Meanwhile, to exploit the complex multi-modal structures of domain adaptation, we use learned features and classifier predictions as the condition to train the adversarial networks. Furthermore, we further propose that the accurately transferable features should enable domain discrepancy to minimum. Three loss functions are introduced into the adversarial networks: classification loss, attention transfer loss, and condition transfer loss. Extensive experiments on Office-31, ImageCLEF-DA, Office-Home, and VisDA-2017 datasets testify that the proposed approach yields state-of-the-art results.