Multi-level supervised hashing with deep features for efficient image retrieval

Abstract Image hashing based on deep convolutional neural networks (CNN), deep hashing, has acquired breakthrough in image retrieval. Although deep features from various CNN layers have various levels of information, most of the existing deep hashing methods extract the feature vector only from the output of the penultimate fully-connected layer, focusing primarily on semantic information whilst ignoring detailed structure information. This calls for research on multi-level hashing, utilizing multi-level features to exploit different levels of CNN characteristics. To fill this gap, a novel image hashing method, Multi-Level Supervised Hashing with deep feature (MLSH), is proposed in this paper to further exploit multiple levels of deep image features. It uses a multiple-hash-table mechanism to integrate multi-level features extracted from an individual deep convolutional neural network. It takes advantage of the complementarity among multi-level features from various layers of a single deep network. High-level features reveal the semantic content of the image, while low-level features provide the structural information that is missing in high-level features. Instead of simple concatenation, several hash tables are trained individually using different levels of features from different layers, which are then integrated for efficient image retrieval. The method has been systematically evaluated through experiments on three image databases, including CIFAR-10, MNIST and NUSWIDE, and has thus been demonstrated to set a new state of the art in image hashing, outperforming several state-of-the-art hashing methods. Furthermore, the recall and precision can be balanced and improved simultaneously.

[1]  Peng Li,et al.  Hashing with dual complementary projection learning for fast image retrieval , 2013, Neurocomputing.

[2]  Svetlana Lazebnik,et al.  Iterative quantization: A procrustean approach to learning binary codes , 2011, CVPR 2011.

[3]  Nicole Immorlica,et al.  Locality-sensitive hashing scheme based on p-stable distributions , 2004, SCG '04.

[4]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[5]  Seungjin Choi,et al.  Sequential Spectral Learning to Hash with Multiple Representations , 2012, ECCV.

[6]  Wenwu Zhu,et al.  Deep Multimodal Hashing with Orthogonal Regularization , 2015, IJCAI.

[7]  Jinhui Tang,et al.  Discriminative Deep Hashing for Scalable Face Image Retrieval , 2017, IJCAI.

[8]  Laith Mohammad Abualigah,et al.  Feature Selection and Enhanced Krill Herd Algorithm for Text Document Clustering , 2018, Studies in Computational Intelligence.

[9]  Yongkang Wong,et al.  A Fine-Grained Spatial-Temporal Attention Model for Video Captioning , 2018, IEEE Access.

[10]  K. T. Talele,et al.  Efficient heterogeneous face recognition using Scale Invariant Feature Transform , 2014, 2014 International Conference on Circuits, Systems, Communication and Information Technology Applications (CSCITA).

[11]  Hongxun Yao,et al.  Exploiting the complementary strengths of multi-layer CNN features for image retrieval , 2017, Neurocomputing.

[12]  Yuxin Peng,et al.  Query-Adaptive Image Retrieval by Deep-Weighted Hashing , 2016, IEEE Transactions on Multimedia.

[13]  Laith Mohammad Abualigah,et al.  Unsupervised text feature selection technique based on hybrid particle swarm optimization algorithm with genetic operators for the text clustering , 2017, The Journal of Supercomputing.

[14]  Trevor Darrell,et al.  Learning to Hash with Binary Reconstructive Embeddings , 2009, NIPS.

[15]  David J. Fleet,et al.  Minimal Loss Hashing for Compact Binary Codes , 2011, ICML.

[16]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[17]  Nicu Sebe,et al.  A Survey on Learning to Hash , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Chun Chen,et al.  Semi-Supervised Nonlinear Hashing Using Bootstrap Sequential Projection Learning , 2013, IEEE Transactions on Knowledge and Data Engineering.

[19]  Fei Wang,et al.  Composite hashing with multiple information sources , 2011, SIGIR.

[20]  M. Omair Ahmad,et al.  Content-based Image Retrieval using Perceptual Image Hashing and Hopfield Neural Network , 2018, 2018 IEEE 61st International Midwest Symposium on Circuits and Systems (MWSCAS).

[21]  Shiguang Shan,et al.  Deep Supervised Hashing for Fast Image Retrieval , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Patrick P. K. Chan,et al.  Asymmetric Cyclical Hashing for Large Scale Image Retrieval , 2015, IEEE Transactions on Multimedia.

[23]  Svetlana Lazebnik,et al.  Locality-sensitive binary codes from shift-invariant kernels , 2009, NIPS.

[24]  Antonio Torralba,et al.  Spectral Hashing , 2008, NIPS.

[25]  Seungjin Choi,et al.  Multi-view anchor graph hashing , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[26]  Kristen Grauman,et al.  Kernelized locality-sensitive hashing for scalable image search , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[27]  Rongrong Ji,et al.  Supervised hashing with kernels , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Larry S. Davis,et al.  Exploiting local features from deep networks for image retrieval , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[29]  Shih-Fu Chang,et al.  Semi-supervised hashing for scalable image retrieval , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Wenmin Wang,et al.  Learning DALTS for cross-modal retrieval , 2019, CAAI Trans. Intell. Technol..

[31]  Svetlana Lazebnik,et al.  Multi-scale Orderless Pooling of Deep Convolutional Activation Features , 2014, ECCV.

[32]  Lei Zhang,et al.  Bit-Scalable Deep Hashing With Regularized Similarity Learning for Image Retrieval and Person Re-Identification , 2015, IEEE Transactions on Image Processing.

[33]  Xuelong Li,et al.  Hierarchical Recurrent Neural Hashing for Image Retrieval With Hierarchical Convolutional Features , 2018, IEEE Transactions on Image Processing.

[34]  Qionghai Dai,et al.  Cross-Modality Bridging and Knowledge Transferring for Image Understanding , 2019, IEEE Transactions on Multimedia.

[35]  Jing Dong,et al.  MFC: A multi-scale fully convolutional approach for visual instance retrieval , 2017, 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[36]  Victor S. Lempitsky,et al.  Aggregating Deep Convolutional Features for Image Retrieval , 2015, ArXiv.

[37]  Hanjiang Lai,et al.  Simultaneous feature learning and hash coding with deep neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Kavita Bala,et al.  Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Hanjiang Lai,et al.  Supervised Hashing for Image Retrieval via Image Representation Learning , 2014, AAAI.

[40]  Li Yan,et al.  Geometric-constrained multi-view image matching method based on semi-global optimization , 2018, Geo spatial Inf. Sci..

[41]  Tao Mei,et al.  Deep Semantic-Preserving and Ranking-Based Hashing for Image Retrieval , 2016, IJCAI.

[42]  Hui Wang,et al.  Weighted multi-deep ranking supervised hashing for efficient image retrieval , 2020, Int. J. Mach. Learn. Cybern..

[43]  Zhi-Hua Zhou,et al.  Column Sampling Based Discrete Supervised Hashing , 2016, AAAI.

[44]  David Stutz,et al.  Neural Codes for Image Retrieval , 2015 .

[45]  Di Huang,et al.  Local Binary Patterns and Its Application to Facial Image Analysis: A Survey , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[46]  Oscar Déniz-Suárez,et al.  Face recognition using Histograms of Oriented Gradients , 2011, Pattern Recognit. Lett..