Kernel based latent semantic sparse hashing for large-scale retrieval from heterogeneous data sources

Abstract Recent years, we have witnessed the growing popularity of integrating nearest neighbor search with hashing for effective and efficient similarity search. However, most of the previous cross-modal hashing methods didn’t consider the semantic correlation between multi-modal representations and directly project the heterogeneous data into a joint space using a linear projection. To address these challenges and bridge the semantic gap more efficiently. We proposed a method named kernel based latent semantic sparse hashing (KLSSH) in this paper. We firstly capture high-level latent semantic information and then use the equivalence between optimizing the code inner products and the Hamming distances. More specifically, KLSSH firstly employs sparse coding for obtaining primary latent features of image and matrix factorization for generating features of text concepts to learn latent semantic features in a high level abstraction space. Next, it maps the latent semantic feature to compact binary codes using kernel method. Kernel scheme ensures to sequentially and efficiently train the hash functions one bit at a time and then generate very short and discriminative hash codes. Moreover, it reduces the quantization loss obviously at the same time and makes the retrieval performance better. Experiments conducted on three benchmark multi-modal datasets demonstrate the superiority of our proposed method compared with the state-of-the-art techniques.