Deep multilevel similarity hashing with fine-grained features for multi-label image retrieval

Abstract For multi-label image retrieval based on deep hashing, the ultimate challenge is to map from the original image to binary space while preserving high-level semantic similarity. Recently, many supervised deep hashing approaches for multi-label image retrieval have been proposed to generate high-quality binary codes. However, most such methods are only introduced to learn simple similarity based on these image characteristics, therein ignoring complex multilevel semantic similarity with fine-grained features. In this paper, we propose a framework named deep hashing with fine-grained feature learning (DH-FFL) to preserve complex multilevel semantic similarity between multi-label image pairs. In this proposed model, compact bilinear pooling convolutional neural networks (CNNs) with normalization are adopted to extract fine-grained feature descriptors. In addition, a novel multilevel contrastive loss is designed to preserve multilevel semantic similarity by introducing a zero-loss parameter. Moreover, a multi-label classification loss is used to maintain the unique semantic structure of each image and maximize the distinguishing ability of binary codes. Comprehensive experiments on three benchmark datasets show that the proposed DH-FFL achieves promising performance compared with other state-of-the-art multi-label image retrieval applications.