Learning Source-Invariant Deep Hashing Convolutional Neural Networks for Cross-Source Remote Sensing Image Retrieval

Due to the urgent demand for remote sensing big data analysis, large-scale remote sensing image retrieval (LSRSIR) attracts increasing attention from researchers. Generally, LSRSIR can be divided into two categories as follows: uni-source LSRSIR (US-LSRSIR) and cross-source LSRSIR (CS-LSRSIR). More specifically, US-LSRSIR means the inquiry remote sensing image and images in the searching data set come from the same remote sensing data source, whereas CS-LSRSIR is designed to retrieve remote sensing images with a similar content to the inquiry remote sensing image that are from a different remote sensing data source. In the literature, US-LSRSIR has been widely exploited, but CS-LSRSIR is rarely discussed. In practical situations, remote sensing images from different kinds of remote sensing data sources are continually increasing, so there is a great motivation to exploit CS-LSRSIR. Therefore, this paper focuses on CS-LSRSIR. To cope with CS-LSRSIR, this paper proposes source-invariant deep hashing convolutional neural networks (SIDHCNNs), which can be optimized in an end-to-end manner using a series of well-designed optimization constraints. To quantitatively evaluate the proposed SIDHCNNs, we construct a dual-source remote sensing image data set that contains eight typical land-cover categories and 10 000 dual samples in each category. Extensive experiments show that the proposed SIDHCNNs can yield substantial improvements over several baselines involving the most recent techniques.

[1]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[2]  Yongjun Zhang,et al.  Content-Based High-Resolution Remote Sensing Image Retrieval via Unsupervised Feature Learning and Collaborative Affinity Metric Fusion , 2016, Remote. Sens..

[3]  Wei Lee Woon,et al.  Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks , 2017 .

[4]  Junwei Han,et al.  Multi-class geospatial object detection and geographic image classification based on collection of part detectors , 2014 .

[5]  Yansheng Li,et al.  Multiple Feature Hashing Learning for Large-Scale Remote Sensing Image Retrieval , 2017, ISPRS Int. J. Geo Inf..

[6]  Chi-Ren Shyu,et al.  Entropy-Balanced Bitmap Tree for Shape-Based Object Retrieval From Large-Scale Satellite Imagery Databases , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Erchan Aptoula,et al.  Remote Sensing Image Retrieval With Global Morphological Texture Descriptors , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification: Benchmark and State of the Art , 2017, Proceedings of the IEEE.

[9]  Dongqing Zhang,et al.  Large-Scale Supervised Multimodal Hashing with Semantic Correlation Maximization , 2014, AAAI.

[10]  Marc Donias,et al.  Structure Tensor Riemannian Statistical Models for CBIR and Classification of Remote Sensing Images , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Wei Zhang,et al.  Robust Feature Matching Method for SAR and Optical Images by Using Gaussian-Gamma-Shaped Bi-Windows-Based Descriptor and Geometric Constraint , 2017, Remote. Sens..

[12]  Albert Y. Zomaya,et al.  Remote sensing big data computing: Challenges and opportunities , 2015, Future Gener. Comput. Syst..

[13]  Wu-Jun Li,et al.  Deep Cross-Modal Hashing , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Shawn D. Newsam,et al.  Geographic Image Retrieval Using Local Invariant Features , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[16]  Yihua Tan,et al.  Unsupervised Multilayer Feature Learning for Satellite Image Scene Classification , 2016, IEEE Geoscience and Remote Sensing Letters.

[17]  Svetlana Lazebnik,et al.  Iterative quantization: A procrustean approach to learning binary codes , 2011, CVPR 2011.

[18]  Jon Atli Benediktsson,et al.  Big Data for Remote Sensing: Challenges and Opportunities , 2016, Proceedings of the IEEE.

[19]  Franz Andert,et al.  Optical aircraft navigation with multi-sensor SLAM and infinite depth features , 2017, 2017 International Conference on Unmanned Aircraft Systems (ICUAS).

[20]  Yanfei Zhong,et al.  A spectral–structural bag-of-features scene classifier for very high spatial resolution remote sensing imagery , 2016 .

[21]  Pietro Perona,et al.  Online crowdsourcing: Rating annotators and obtaining cost-effective labels , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[22]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[23]  Xin Huang,et al.  Unsupervised Deep Feature Learning for Urban Village Detection from High-Resolution Remote Sensing Images , 2017 .

[24]  Shiguang Shan,et al.  Deep Supervised Hashing for Fast Image Retrieval , 2016, International Journal of Computer Vision.

[25]  Bin Luo,et al.  Indexing of Satellite Images With Different Resolutions by Wavelet Features , 2008, IEEE Transactions on Image Processing.

[26]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[27]  Naif Alajlan,et al.  Domain Adaptation Network for Cross-Scene Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Jing Li,et al.  Cross-Domain Co-Occurring Feature for Visible-Infrared Image Matching , 2018, IEEE Access.

[29]  Lorenzo Bruzzone,et al.  Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[30]  Yongjun Zhang,et al.  Large-Scale Remote Sensing Image Retrieval by Deep Hashing Neural Networks , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Fan Mo,et al.  Matching Multi-Source Optical Satellite Imagery Exploiting a Multi-Stage Approach , 2017, Remote. Sens..

[32]  Zhi-Hua Zhou,et al.  Column Sampling Based Discrete Supervised Hashing , 2016, AAAI.

[33]  Chi-Ren Shyu,et al.  GeoIRIS: Geospatial Information Retrieval and Indexing System—Content Mining, Semantics Modeling, and Complex Queries , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Min-Fan Ricky Lee,et al.  Visual navigation system for small unmanned aerial vehicles , 2013 .

[35]  B. Bhaduri,et al.  Automated Feature Generation in Large-Scale Geospatial Libraries for Content-Based Indexing. , 2006 .

[36]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[37]  Jianmin Wang,et al.  Deep Hashing Network for Efficient Similarity Retrieval , 2016, AAAI.

[38]  Lorenzo Bruzzone,et al.  Hashing-Based Scalable Remote Sensing Image Search and Retrieval in Large Archives , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Xiang Li,et al.  Full-parameter vision navigation based on scene matching for aircrafts , 2014, Science China Information Sciences.

[40]  Peng Ren,et al.  Partial Randomness Hashing for Large-Scale Remote Sensing Image Retrieval , 2017, IEEE Geoscience and Remote Sensing Letters.

[41]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[42]  Supratik Mukhopadhyay,et al.  DeepSat: a learning framework for satellite imagery , 2015, SIGSPATIAL/GIS.

[43]  William J. Emery,et al.  Contextually guided very-high-resolution imagery classification with semantic segments , 2017 .