Semantic Hierarchy Preserving Deep Hashing for Large-scale Image Retrieval

Convolutional neural networks have been widely used in content-based image retrieval. To better deal with large-scale data, the deep hashing model is proposed as an effective method, which maps an image to a binary code that can be used for hashing search. However, most existing deep hashing models only utilize fine-level semantic labels or convert them to similar/dissimilar labels for training. The natural semantic hierarchy structures are ignored in the training stage of the deep hashing model. In this paper, we present an effective algorithm to train a deep hashing model that can preserve a semantic hierarchy structure for large-scale image retrieval. Experiments on two datasets show that our method improves the fine-level retrieval performance. Meanwhile, our model achieves state-of-the-art results in terms of hierarchical retrieval.

[1]  Fei-Fei Li,et al.  Hierarchical semantic indexing for large scale image retrieval , 2011, CVPR 2011.

[2]  Dan Wang,et al.  Supervised Deep Hashing for Hierarchical Labeled Data , 2017, AAAI.

[3]  Xiang Cheng,et al.  Deep Supervised Hashing with Nonlinear Projections , 2017, IJCAI.

[4]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[5]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Shifeng Chen,et al.  Deep center-based dual-constrained hashing for discriminative face image retrieval , 2021, Pattern Recognit..

[7]  Wei Liu,et al.  Sub-Selective Quantization for Large-Scale Image Search , 2014, AAAI.

[8]  Minho Lee,et al.  Content-based image retrieval by using deep kernel Machine with Gaussian Mixture Model , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[9]  Svetlana Lazebnik,et al.  Iterative quantization: A procrustean approach to learning binary codes , 2011, CVPR 2011.

[10]  Joachim Denzler,et al.  Hierarchy-Based Image Embeddings for Semantic Image Retrieval , 2018, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[11]  Jiashi Feng,et al.  Central Similarity Quantization for Efficient Image and Video Retrieval , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[13]  Hao Zhang,et al.  A Survey on Deep Hashing Methods , 2020, ArXiv.

[14]  Michael Healy,et al.  Theory and Applications of Ontology: Computer Applications , 2010 .

[15]  Prateek Jain,et al.  Fast Similarity Search for Learned Metrics , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Georges Quénot,et al.  Improving Image Classification using Coarse and Fine Labels , 2017, ICMR.

[17]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[18]  Philip S. Yu,et al.  HashNet: Deep Learning to Hash by Continuation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[19]  Tao Mei,et al.  Deep Semantic-Preserving and Ranking-Based Hashing for Image Retrieval , 2016, IJCAI.

[20]  Wu-Jun Li,et al.  Feature Learning Based Deep Supervised Hashing with Pairwise Labels , 2015, IJCAI.

[21]  Shiguang Shan,et al.  Hierarchical Training for Large Scale Face Recognition with Few Samples Per Subject , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[22]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[23]  Chu-Song Chen,et al.  Supervised Learning of Semantics-Preserving Hash via Deep Convolutional Neural Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Hong Yan,et al.  Deep Class-Wise Hashing: Semantics-Preserving Hashing via Class-Wise Loss , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Ming Zhang,et al.  Improved Deep Classwise Hashing With Centers Similarity Learning for Image Retrieval , 2021, 2020 25th International Conference on Pattern Recognition (ICPR).

[26]  Jianmin Wang,et al.  Deep Quantization Network for Efficient Image Retrieval , 2016, AAAI.

[27]  Zheng Zhang,et al.  MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems , 2015, ArXiv.

[28]  Tieniu Tan,et al.  Deep Supervised Discrete Hashing , 2017, NIPS.

[29]  Yi Shi,et al.  Deep Supervised Hashing with Triplet Labels , 2016, ACCV.