Asymmetric Hash Code Learning for Remote Sensing Image Retrieval

Remote sensing image retrieval (RSIR), aiming at searching for a set of similar items to a given query image, is a very important task in remote sensing applications. Deep hashing learning as the current mainstream method has achieved satisfactory retrieval performance. On one hand, various deep neural networks are used to extract semantic features of remote sensing images. On the other hand, the hashing techniques are subsequently adopted to map the high-dimensional deep features to the low-dimensional binary codes. This kind of methods attempts to learn one hash function for both the query and database samples in a symmetric way. However, with the number of database samples increasing, it is typically timeconsuming to generate the hash codes of large-scale database images. In this paper, we propose a novel deep hashing method, named asymmetric hash code learning (AHCL), for RSIR. The proposed AHCL generates the hash codes of query and database images in an asymmetric way. In more detail, the hash codes of query images are obtained by binarizing the output of the network, while the hash codes of database images are directly learned by solving the designed objective function. In addition, we combine the semantic information of each image and the similarity information of pairs of images as supervised information to train a deep hashing network, which improves the representation ability of deep features and hash codes. The experimental results on three public datasets demonstrate that the proposed method outperforms symmetric methods in terms of retrieval accuracy and efficiency. The source code is available at https://github.com/weiweisong415/Demo AHCL for TGRS2022.

[1]  Anthony Stefanidis,et al.  An environment for content-based image retrieval from large spatial databases , 1999 .

[2]  Shutao Li,et al.  Deep Hyperspectral Image Sharpening , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Medeni Soysal,et al.  Performance Analysis of State-of-the-Art Representation Methods for Geographical Image Retrieval and Categorization , 2014, IEEE Geoscience and Remote Sensing Letters.

[4]  Shutao Li,et al.  Hyperspectral Image Classification With Deep Feature Fusion Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Heng Tao Shen,et al.  Hashing for Similarity Search: A Survey , 2014, ArXiv.

[6]  Jocelyn Chanussot,et al.  Multimodal remote sensing benchmark datasets for land cover classification with a shared and specific feature learning model , 2021, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[7]  Fang Liu,et al.  Large-Scale Remote Sensing Image Retrieval Based on Semi-Supervised Adversarial Hashing , 2019, Remote. Sens..

[8]  Weiwei Song,et al.  Deep Hashing Neural Networks for Hyperspectral Image Feature Extraction , 2019, IEEE Geoscience and Remote Sensing Letters.

[9]  Lianru Gao,et al.  Graph Convolutional Networks for Hyperspectral Image Classification , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Wen Yang,et al.  STRUCTURAL HIGH-RESOLUTION SATELLITE IMAGE INDEXING , 2010 .

[11]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Lorenzo Bruzzone,et al.  Hashing-Based Scalable Remote Sensing Image Search and Retrieval in Large Archives , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Jocelyn Chanussot,et al.  Graph Relation Network: Modeling Relations Between Scenes for Multilabel Remote-Sensing Image Classification and Retrieval , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Bin Sun,et al.  Recent Advances and New Guidelines on Hyperspectral and Multispectral Image Fusion , 2020, Inf. Fusion.

[15]  Renwei Dian,et al.  Regularizing Hyperspectral and Multispectral Image Fusion by CNN Denoiser , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Jon Atli Benediktsson,et al.  Deep Learning for Hyperspectral Image Classification: An Overview , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Mihai Datcu,et al.  Exploiting Deep Features for Remote Sensing Image Retrieval: A Systematic Investigation , 2017, IEEE Transactions on Big Data.

[18]  Xueliang Zhang,et al.  Deep learning in remote sensing applications: A meta-analysis and review , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[19]  Jon Atli Benediktsson,et al.  Deep Hashing Learning for Visual and Semantic Retrieval of Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Licheng Jiao,et al.  Deep Hash Learning for Remote Sensing Image Retrieval , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Odej Kao,et al.  Retrieval of remotely sensed imagery using spectral information content , 2002, IEEE International Geoscience and Remote Sensing Symposium.

[22]  Antonio Torralba,et al.  Spectral Hashing , 2008, NIPS.

[23]  Peng Ren,et al.  Partial Randomness Hashing for Large-Scale Remote Sensing Image Retrieval , 2017, IEEE Geoscience and Remote Sensing Letters.

[24]  Pingping Liu,et al.  Deep Hash Remote Sensing Image Retrieval with Hard Probability Sampling , 2020, Remote. Sens..

[25]  Shutao Li,et al.  Remote Sensing Scene Classification Using Multilayer Stacked Covariance Pooling , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Gui-Song Xia,et al.  AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Shawn D. Newsam,et al.  Geographic Image Retrieval Using Local Invariant Features , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[29]  Rongrong Ji,et al.  Supervised hashing with kernels , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Zhenfeng Shao,et al.  PatternNet: A Benchmark Dataset for Performance Evaluation of Remote Sensing Image Retrieval , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[31]  Wu-Jun Li,et al.  Asymmetric Deep Supervised Hashing , 2017, AAAI.

[32]  Naoto Yokoya,et al.  An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing , 2018, IEEE Transactions on Image Processing.

[33]  Wu-Jun Li,et al.  Feature Learning Based Deep Supervised Hashing with Pairwise Labels , 2015, IJCAI.

[34]  Chi-Ren Shyu,et al.  Entropy-Balanced Bitmap Tree for Shape-Based Object Retrieval From Large-Scale Satellite Imagery Databases , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Shutao Li,et al.  Learning a Low Tensor-Train Rank Representation for Hyperspectral Image Super-Resolution , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Christos Grecos,et al.  Cohesion Intensive Deep Hashing for Remote Sensing Image Retrieval , 2019, Remote. Sens..

[37]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[38]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[39]  Zi Huang,et al.  A Sparse Embedding and Least Variance Encoding Approach to Hashing , 2014, IEEE Transactions on Image Processing.

[40]  Yongjun Zhang,et al.  Large-Scale Remote Sensing Image Retrieval by Deep Hashing Neural Networks , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Deng Cai,et al.  Density Sensitive Hashing , 2012, IEEE Transactions on Cybernetics.

[42]  Ling Shao,et al.  Unsupervised Deep Hashing With Pseudo Labels for Scalable Image Retrieval , 2018, IEEE Transactions on Image Processing.

[43]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[44]  Julie Delon,et al.  Shape-based Invariant Texture Indexing , 2010, International Journal of Computer Vision.

[45]  Hanjiang Lai,et al.  Supervised Hashing for Image Retrieval via Image Representation Learning , 2014, AAAI.

[46]  Naoto Yokoya,et al.  More Diverse Means Better: Multimodal Deep Learning Meets Remote-Sensing Imagery Classification , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[47]  Yongjun Zhang,et al.  Learning Source-Invariant Deep Hashing Convolutional Neural Networks for Cross-Source Remote Sensing Image Retrieval , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[48]  Shawn D. Newsam,et al.  Learning Low Dimensional Convolutional Neural Networks for High-Resolution Remote Sensing Image Retrieval , 2016, Remote. Sens..

[49]  Svetlana Lazebnik,et al.  Iterative quantization: A procrustean approach to learning binary codes , 2011, CVPR 2011.

[50]  Paolo Napoletano,et al.  Visual descriptors for content-based retrieval of remote-sensing images , 2016, ArXiv.