High-Resolution Remote Sensing Image Scene Classification via Key Filter Bank Based on Convolutional Neural Network

High-resolution remote sensing (HRRS) image scene classification has attracted an enormous amount of attention due to its wide application in a range of tasks. Due to the rapid development of deep learning (DL), models based on convolutional neural network (CNN) have made competitive achievements on HRRS image scene classification because of the excellent representation capacity of DL. The scene labels of HRRS images extremely depend on the combination of global information and information from key regions or locations. However, most existing models based on CNN tend only to represent the global features of images or overstate local information capturing from key regions or locations, which may confuse different categories. To address this issue, a key region or location capturing method called key filter bank (KFB) is proposed in this article, and KFB can retain global information at the same time. This method can combine with different CNN models to improve the performance of HRRS imagery scene classification. Moreover, for the convenience of practical tasks, an end-to-end model called KFBNet where KFB combined with DenseNet-121 is proposed to compare the performance with existing models. This model is evaluated on public benchmark data sets, and the proposed model makes better performance on benchmarks than the state-of-the-art methods.

[1]  Bo Huang,et al.  Urban land-use mapping using a deep convolutional neural network with high spatial resolution multispectral remote sensing imagery , 2018, Remote Sensing of Environment.

[2]  Ching Y. Suen,et al.  Scene Classification Using Hierarchical Wasserstein CNN , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[3]  M. Schaepman,et al.  Robust quantification of riverine land cover dynamics by high-resolution remote sensing , 2018, Remote Sensing of Environment.

[4]  Vladimir Risojevic,et al.  Fusion of Global and Local Descriptors for Remote Sensing Image Classification , 2013, IEEE Geoscience and Remote Sensing Letters.

[5]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[6]  Liangpei Zhang,et al.  Unsupervised Change Detection Based on Hybrid Conditional Random Field Model for High Spatial Resolution Remote Sensing Imagery , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Jie Cheng,et al.  Using Very High Resolution Thermal Infrared Imagery for More Accurate Determination of the Impact of Land Cover Differences on Evapotranspiration in an Irrigated Agricultural Area , 2019, Remote. Sens..

[8]  Xiaogang Wang,et al.  Residual Attention Network for Image Classification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Peijun Du,et al.  Object-Based Change Detection in Urban Areas from High Spatial Resolution Images Based on Multiple Features and Ensemble Learning , 2018, Remote. Sens..

[10]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[11]  Qian Du,et al.  Learning Sensor-Specific Spatial-Spectral Features of Hyperspectral Images via Convolutional Neural Networks , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Lei Guo,et al.  Remote Sensing Image Scene Classification Using Bag of Convolutional Features , 2017, IEEE Geoscience and Remote Sensing Letters.

[13]  Chao Huang,et al.  Scene Classification via Triplet Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[14]  Anil M. Cheriyadat,et al.  Unsupervised Feature Learning for Aerial Scene Classification , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Jefersson Alex dos Santos,et al.  Towards better exploiting convolutional neural networks for remote sensing scene classification , 2016, Pattern Recognit..

[16]  Xuelong Li,et al.  A Hybrid Sparsity and Distance-Based Discrimination Detector for Hyperspectral Images , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Naif Alajlan,et al.  Using convolutional features and a sparse autoencoder for land-use scene classification , 2016 .

[18]  Rodrigo Minetto,et al.  Hydra: An Ensemble of Convolutional Neural Networks for Geospatial Land Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[19]  B. S. Manjunath,et al.  Modeling and Detection of Geospatial Objects Using Texture Motifs , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Jürgen Schmidhuber,et al.  Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction , 2011, ICANN.

[21]  Lizhe Wang,et al.  A semi-supervised generative framework with deep learning features for high-resolution remote sensing image scene classification , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[22]  Jonathan Cheung-Wai Chan,et al.  Hyperspectral Images Classification Based on Dense Convolutional Networks with Spectral-Wise Attention Mechanism , 2019, Remote. Sens..

[23]  Antonio Plaza,et al.  Scale-Free Convolutional Neural Network for Remote Sensing Scene Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[24]  Gui-Song Xia,et al.  Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery , 2015, Remote. Sens..

[25]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[26]  Jinfang Zhang,et al.  Land use and land cover classification base on image saliency map cooperated coding , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[27]  Qian Du,et al.  Fusing Local and Global Features for High-Resolution Scene Classification , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[28]  Yihua Tan,et al.  Unsupervised Multilayer Feature Learning for Satellite Image Scene Classification , 2016, IEEE Geoscience and Remote Sensing Letters.

[29]  Libao Zhang,et al.  Airport Detection and Aircraft Recognition Based on Two-Layer Saliency Model in High Spatial Resolution Remote-Sensing Images , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[30]  Lei Guo,et al.  Effective and Efficient Midlevel Visual Elements-Oriented Land-Use Classification Using VHR Remote Sensing Images , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[32]  Hongxun Yao,et al.  Deep Feature Fusion for VHR Remote Sensing Scene Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[33]  Gui-Song Xia,et al.  Dirichlet-Derived Multiple Topic Scene Classification Model for High Spatial Resolution Remote Sensing Imagery , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Yunlong Yu,et al.  Dense Connectivity Based Two-Stream Deep Feature Fusion Framework for Aerial Scene Classification , 2018, Remote. Sens..

[35]  Junwei Han,et al.  Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[37]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[39]  Gui-Song Xia,et al.  Bag-of-Visual-Words Scene Classifier With Local and Global Features for High Spatial Resolution Remote Sensing Imagery , 2016, IEEE Geoscience and Remote Sensing Letters.

[40]  Lei Guo,et al.  Learning coarse-to-fine sparselets for efficient object detection and scene classification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[42]  Bo Du,et al.  Saliency-Guided Unsupervised Feature Learning for Scene Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[43]  Fraser Baker,et al.  A Combined Approach to Classifying Land Surface Cover of Urban Domestic Gardens Using Citizen Science Data and High Resolution Image Analysis , 2018, Remote. Sens..

[44]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[45]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[46]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[47]  Xiangtao Zheng,et al.  Remote Sensing Scene Classification by Unsupervised Representation Learning , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[48]  Qiang Zhang,et al.  Domain Adaptation for Convolutional Neural Networks-Based Remote Sensing Scene Classification , 2019, IEEE Geoscience and Remote Sensing Letters.

[49]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Junwei Han,et al.  Automatic landslide detection from remote-sensing imagery using a scene classification method based on BoVW and pLSA , 2013 .

[51]  Lei Guo,et al.  When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[52]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Xuelong Li,et al.  Scene Classification With Recurrent Attention of VHR Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[54]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[55]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Gui-Song Xia,et al.  AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[57]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Naif Alajlan,et al.  Land-Use Classification With Compressive Sensing Multifeature Fusion , 2015, IEEE Geoscience and Remote Sensing Letters.

[60]  Chuan-Yu Chang,et al.  A CNN-Based Method for Infant Cry Detection and Recognition , 2019, AINA Workshops.

[61]  Cong Lin,et al.  Integrating Multilayer Features of Convolutional Neural Networks for Remote Sensing Scene Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[62]  Hong Tang,et al.  DropBand: A Simple and Effective Method for Promoting the Scene Classification Accuracy of Convolutional Neural Networks for VHR Remote Sensing Imagery , 2018, IEEE Geoscience and Remote Sensing Letters.

[63]  Fahad Shahbaz Khan,et al.  Binary Patterns Encoded Convolutional Neural Networks for Texture Recognition and Remote Sensing Scene Classification , 2017, ArXiv.

[64]  Liang Chen,et al.  IORN: An Effective Remote Sensing Image Scene Classification Framework , 2018, IEEE Geoscience and Remote Sensing Letters.

[65]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[66]  Xiangtao Zheng,et al.  Exploring Models and Data for Remote Sensing Image Caption Generation , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[67]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[68]  Qian Du,et al.  Scene classification using local and global features with collaborative representation fusion , 2016, Inf. Sci..

[69]  Lei Guo,et al.  Object Detection in Optical Remote Sensing Images Based on Weakly Supervised Learning and High-Level Feature Learning , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[70]  Shawn D. Newsam,et al.  Geographic Image Retrieval Using Local Invariant Features , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[71]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[72]  Tao Zhang,et al.  Deep topology network: A framework based on feedback adjustment learning rate for image classification , 2019, Adv. Eng. Informatics.

[73]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification Using Rearranged Local Features , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[74]  T. Charles Clancy,et al.  Convolutional Radio Modulation Recognition Networks , 2016, EANN.

[75]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[76]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[77]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.