Hierarchical deep transfer learning for fine-grained categorization on micro datasets

Abstract Fine-grained categorization is challenging due to its small inter-class and large intra-class variance. Moreover, requiring domain expertise makes fine-grained labelled data much more expensive to acquire. Existing models predominantly require extra information such as bounding box and part annotation in addition to the image category labels, which involves heavy manual labor. In this paper, we propose a novel hierarchical deep transfer learning model, based on a compression convolutional neural network. Our model transfers the learned image representation from large-scale labelled fine-grained datasets to micro fine-grained datasets, which avoids using expensive annotations and realizes visual categorization task effectively. Firstly, we introduce a cohesion domain to measure correlation degree between source domain and target domain. Secondly, the source-domain convolutional neural network is adjusted according to its metrical feedback, in order to select task-specific features that are suitable for transferring to the target domain. Finally, we make most of perspective-class labels, which are inherent attributes of fine-grained data for multi-task learning and learn all the attributes through joint learning to extract more discriminative representations. The proposed model not only economizes training time effectively and achieves high categorization accuracy, but also verifies that the inter-domain feature transition can accelerate learning and optimization.