Global and local semantics-preserving based deep hashing for cross-modal retrieval

Abstract Cross-modal hashing methods map similar data entities from heterogeneous data sources to binary codes with smaller Hamming distance. However, most existing cross-modal hashing methods learn the hash codes with the hand-crafted features which will not generate optimal hash codes and achieve satisfactory performance. Deep cross-modal hashing methods integrate feature learning and hash coding into an end-to-end learning framework which have achieved promising results. However, these deep cross-modal hashing methods do not well preserve the discriminative ability and the global multilevel similarity in hash learning procedure. In this paper, we propose a global and local semantics-preserving based deep hashing method for cross-modal retrieval. More specifically, a large margin is enforced between similar hash codes and dissimilar hash codes from an inter-modal view to learn discriminative hash codes. Therefore the learned hash codes can well preserve local semantic structure. Sequently, the supervised information with the global multilevel similarity is introduced to learn semantics-preserving hash codes for each intra-modal view. As a consequence, the global semantic structure can be preserved into the hash codes. Furthermore, a consistent regularization constraint is added to generate unified hash codes. Ultimately, the feature learning procedure and the hash coding procedure are integrated into an end-to-end learning framework. To verify the effectiveness of the proposed method, extensive experiments are conducted on several datasets, and the experimental results demonstrate that the proposed method achieves superior performance.