Improving the Generalization Ability of Deep Neural Networks for Cross-Domain Visual Recognition

Feature learning with deep neural networks has made remarkable progress in recent years. However, its data-driven nature makes the collection of labeled training data expensive or impossible when the testing domain changes. Here we propose a method of transferable feature learning and instance-level adaptation to improve the generalization ability of deep neural networks so as to mitigate the domain shift challenge for cross-domain visual recognition. When less labeled information is available, our proposed method shows attractive results in the new target domain and outperforms the typical fine-tuning method. Two deep neural networks are chosen as the representatives to be further developed with our proposed method, to do a comprehensive study about the generalization ability on the tasks of image-to-image transfer, image-to-video transfer, multi-domain image classification and weakly supervised detection. Experimental results show that our proposed method is superior to other existing works in the literature. In addition, a large scale of cross-domain database is merged from three different domains, providing a quantitative platform to evaluate different approaches in the field of cross-domain object detection.