Towards Joint Multiply Semantics Hashing for Visual Search

With the rapid growth of visual data on the web, deep hashing has shown enormous potential in preserving semantic similarity for visual search. Currently, most of the existing hashing methods employ pairwise or triplet-wise constraint to obtain the semantic similarity or relatively similarity among binary codes. However, some potential semantic context cannot be fully exploited, resulting in a suboptimal visual search. In this paper, we propose a novel deep hashing method, termed Joint Multiply Semantics Hashing (JMSH), to learn discriminative yet compact binary codes. In our approach, We jointly learn multiply semantic information to perform feature learning and hash coding. To be specific, the semantic information includes the pairwise semantic similarity between binary codes, the pointwise binary codes semantics and the pointwise visual feature semantics. Meanwhile, three different loss functions are designed to train the JMSH model. Extensive experiments show that the proposed JMSH yields state-of-the-art retrieval performance on representative image retrieval benchmarks.

[1]  Meng Wang,et al.  Neighborhood Discriminant Hashing for Large-Scale Image Retrieval , 2015, IEEE Transactions on Image Processing.

[2]  Liqiang Nie,et al.  Fast Scalable Supervised Hashing , 2018, SIGIR.

[3]  Wu-Jun Li,et al.  Feature Learning Based Deep Supervised Hashing with Pairwise Labels , 2015, IJCAI.

[4]  Wei Liu,et al.  Discrete Graph Hashing , 2014, NIPS.

[5]  Hanjiang Lai,et al.  Simultaneous feature learning and hash coding with deep neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Philip S. Yu,et al.  HashNet: Deep Learning to Hash by Continuation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[7]  Jun Wang,et al.  Self-taught hashing for fast similarity search , 2010, SIGIR.

[8]  Kristen Grauman,et al.  Kernelized Locality-Sensitive Hashing , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Svetlana Lazebnik,et al.  Locality-sensitive binary codes from shift-invariant kernels , 2009, NIPS.

[10]  Svetlana Lazebnik,et al.  Iterative quantization: A procrustean approach to learning binary codes , 2011, CVPR 2011.

[11]  Zhenan Sun,et al.  Local Semantic-Aware Deep Hashing With Hamming-Isometric Quantization , 2019, IEEE Transactions on Image Processing.

[12]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Wei Liu,et al.  Learning Binary Codes for Maximum Inner Product Search , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Chu-Song Chen,et al.  Supervised Learning of Semantics-Preserving Hash via Deep Convolutional Neural Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Yang Yang,et al.  Deep Asymmetric Pairwise Hashing , 2017, ACM Multimedia.

[17]  Piotr Indyk,et al.  Similarity Search in High Dimensions via Hashing , 1999, VLDB.

[18]  Hanjiang Lai,et al.  Supervised Hashing for Image Retrieval via Image Representation Learning , 2014, AAAI.

[19]  Jianmin Wang,et al.  Deep Hashing Network for Efficient Similarity Retrieval , 2016, AAAI.

[20]  Cordelia Schmid,et al.  Product Quantization for Nearest Neighbor Search , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  David J. Fleet,et al.  Minimal Loss Hashing for Compact Binary Codes , 2011, ICML.

[22]  Antonio Torralba,et al.  Spectral Hashing , 2008, NIPS.

[23]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[24]  Minyi Guo,et al.  Supervised hashing with latent factor models , 2014, SIGIR.

[25]  Shiguang Shan,et al.  Deep Supervised Hashing for Fast Image Retrieval , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Svetlana Lazebnik,et al.  Iterative quantization: A procrustean approach to learning binary codes , 2011, CVPR 2011.

[27]  Lei Zhang,et al.  Bit-Scalable Deep Hashing With Regularized Similarity Learning for Image Retrieval and Person Re-Identification , 2015, IEEE Transactions on Image Processing.

[28]  Nicu Sebe,et al.  A Survey on Learning to Hash , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Philip S. Yu,et al.  Deep Priority Hashing , 2018, ACM Multimedia.

[30]  Tieniu Tan,et al.  Deep Supervised Discrete Hashing , 2017, NIPS.

[31]  Wei Liu,et al.  Supervised Discrete Hashing , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  David Suter,et al.  Fast Supervised Hashing with Decision Trees for High-Dimensional Data , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Jiwen Lu,et al.  Learning Compact Binary Descriptors with Unsupervised Deep Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).