Scene classification of high-resolution remote sensing images based on IMFNet

Abstract. Currently, due to the limited amount of data and the difficulty of designing a network, there are few papers on constructing a new convolutional neural network for scene classification using the publicly available datasets of high-resolution remote sensing images. Considering the existing problems, the current scene classification methods of high-resolution remote sensing images are summarized, and the IMFNet model is constructed to classify scenes of high-resolution remote sensing images in this paper. The IMFNet is an end-to-end network, which can learn features from data automatically. The main characteristic of the IMFNet network structure is that the Inception module is used to extract the details of remote sensing images and the multifeature fusion strategy is proposed to ensure the integrity of information. In addition, optimization methods are adopted to improve the classification accuracy. In order to verify the effectiveness of the method proposed in this paper, the two benchmark datasets—the UC Merced dataset and the SIRI-WHU dataset were adopted for experiments. The classification accuracy of the two datasets reaches 92.14% and 90.43%, respectively. Experimental results show that the method proposed has certain advantages over the classification methods based on low-level and middle-level visual features and even some classification methods based on high-level visual features.

[1]  Zhong Xie,et al.  An Improved Bag-of-Visual-Word Based Classification Method for High-Resolution Remote Sensing Scene , 2018, 2018 26th International Conference on Geoinformatics.

[2]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[3]  Bo Du,et al.  Scene Classification via a Gradient Boosting Random Convolutional Network Framework , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Geoffrey E. Hinton,et al.  An Efficient Learning Procedure for Deep Boltzmann Machines , 2012, Neural Computation.

[5]  Zhang Liangpei,et al.  Automatic Analysis and Mining of Remote Sensing Big Data , 2014 .

[6]  Lei Guo,et al.  Effective and Efficient Midlevel Visual Elements-Oriented Land-Use Classification Using VHR Remote Sensing Images , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Lizhe Wang,et al.  Adaptive Spatial-Scale-Aware Deep Convolutional Neural Network for High-Resolution Remote Sensing Imagery Scene Classification , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[8]  Liangpei Zhang,et al.  A Deep-Local-Global Feature Fusion Framework for High Spatial Resolution Imagery Scene Classification , 2018, Remote. Sens..

[9]  Liangpei Zhang,et al.  Pre-Trained AlexNet Architecture with Pyramid Pooling and Supervision for High Spatial Resolution Remote Sensing Image Scene Classification , 2017, Remote. Sens..

[10]  Muhammad Awais,et al.  Medical image retrieval using deep convolutional neural network , 2017, Neurocomputing.

[11]  Zhenfeng Shao,et al.  PatternNet: A Benchmark Dataset for Performance Evaluation of Remote Sensing Image Retrieval , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[12]  Xuan Liu,et al.  Classifying High Resolution Remote Sensing Images by Fine-Tuned VGG Deep Networks , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[13]  Weihong Deng,et al.  Very deep convolutional neural network based image classification using small training sample size , 2015, 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR).

[14]  Junwei Han,et al.  Automatic landslide detection from remote-sensing imagery using a scene classification method based on BoVW and pLSA , 2013 .

[15]  Shiyong Cui,et al.  A Comparative Study of Bag-of-Words and Bag-of-Topics Models of EO Image Patches , 2015, IEEE Geoscience and Remote Sensing Letters.

[16]  Ping Tang,et al.  Feature significance-based multibag-of-visual-words model for remote sensing image scene classification , 2016 .

[17]  Supratik Mukhopadhyay,et al.  DeepSat: a learning framework for satellite imagery , 2015, SIGSPATIAL/GIS.

[18]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[19]  Bo Du,et al.  Saliency-Guided Unsupervised Feature Learning for Scene Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Yongjun Zhang,et al.  Content-Based High-Resolution Remote Sensing Image Retrieval via Unsupervised Feature Learning and Collaborative Affinity Metric Fusion , 2016, Remote. Sens..

[21]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[23]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[24]  Zenghui Wang,et al.  Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review , 2017, Neural Computation.

[25]  Chao Zeng,et al.  Missing Data Reconstruction in Remote Sensing Image With a Unified Spatial–Temporal–Spectral Deep Convolutional Neural Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[27]  ZissermanAndrew,et al.  Scene Classification Using a Hybrid Generative/Discriminative Approach , 2008 .

[28]  Luisa Verdoliva,et al.  Land Use Classification in Remote Sensing Images by Convolutional Neural Networks , 2015, ArXiv.

[29]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[30]  Mihai Datcu,et al.  Semantic Annotation of Satellite Images Using Latent Dirichlet Allocation , 2010, IEEE Geoscience and Remote Sensing Letters.

[31]  Anil K. Jain,et al.  Object detection using gabor filters , 1997, Pattern Recognit..

[32]  Niti B. Mishra,et al.  Mapping vegetation morphology types in a dry savanna ecosystem: integrating hierarchical object-based image analysis with Random Forest , 2014 .

[33]  Bei Zhao,et al.  Scene classification based on a hierarchical convolutional sparse auto-encoder for high spatial resolution imagery , 2017 .

[34]  Uwe Stilla,et al.  Deep Learning Earth Observation Classification Using ImageNet Pretrained Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[35]  Guofang Lv,et al.  Integration of heterogeneous features for remote sensing scene classification , 2018 .

[36]  Junwei Han,et al.  Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[39]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[40]  Gui-Song Xia,et al.  AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Jinhua Xu,et al.  Object-Based Representation for Scene Classification , 2016, Canadian Conference on AI.

[42]  Yuning Jiang,et al.  Randomized Spatial Partition for Scene Recognition , 2012, ECCV.

[43]  Hong Sun,et al.  Unsupervised feature coding on local patch manifold for satellite image scene classification , 2014, 2014 IEEE Geoscience and Remote Sensing Symposium.

[44]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[46]  Dengxin Dai,et al.  Satellite Image Classification via Two-Layer Sparse Coding With Biased Image Representation , 2011, IEEE Geoscience and Remote Sensing Letters.

[47]  Gong Cheng,et al.  Scene classification of high resolution remote sensing images using convolutional neural networks , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[48]  Yanfei Liu,et al.  Scene Classification Based on a Deep Random-Scale Stretched Convolutional Neural Network , 2018, Remote. Sens..

[49]  Shawn D. Newsam,et al.  Spatial pyramid co-occurrence for image classification , 2011, 2011 International Conference on Computer Vision.

[50]  Yuan Hu,et al.  A Sample Update-Based Convolutional Neural Network Framework for Object Detection in Large-Area Remote Sensing Images , 2019, IEEE Geoscience and Remote Sensing Letters.

[51]  Yuanyuan Liu,et al.  Deep Salient Feature Based Anti-Noise Transfer Network for Scene Classification of Remote Sensing Imagery , 2018, Remote. Sens..

[52]  Andrew Zisserman,et al.  Scene Classification Using a Hybrid Generative/Discriminative Approach , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[54]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[55]  Yanfei Zhong,et al.  Large patch convolutional neural networks for the scene classification of high spatial resolution imagery , 2016 .

[56]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[57]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification: Benchmark and State of the Art , 2017, Proceedings of the IEEE.

[58]  Jefersson Alex dos Santos,et al.  Do deep features generalize from everyday objects to remote sensing and aerial scenes domains? , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[59]  Bidyut Baran Chaudhuri,et al.  A survey of Hough Transform , 2015, Pattern Recognit..

[60]  Gui-Song Xia,et al.  Dirichlet-Derived Multiple Topic Scene Classification Model for High Spatial Resolution Remote Sensing Imagery , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[61]  Brian P. Salmon,et al.  Multiview Deep Learning for Land-Use Classification , 2015, IEEE Geoscience and Remote Sensing Letters.

[62]  Qianqing Qin,et al.  Scene Classification Based on Multiscale Convolutional Neural Network , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[63]  Bolei Zhou,et al.  Learning Deep Features for Scene Recognition using Places Database , 2014, NIPS.

[64]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[65]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[66]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[67]  Liangpei Zhang,et al.  The Fisher Kernel Coding Framework for High Spatial Resolution Scene Classification , 2016, Remote. Sens..

[68]  Vladimir Risojevic,et al.  Gabor Descriptors for Aerial Image Classification , 2011, ICANNGA.

[69]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[70]  Tong Zhang,et al.  Deep Learning Based Feature Selection for Remote Sensing Scene Classification , 2015, IEEE Geoscience and Remote Sensing Letters.

[71]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[72]  G. Wilkinson,et al.  Forest mapping from multi‐source satellite data using neural network classifiers—an experiment in Portugal , 1995 .

[73]  Gui-Song Xia,et al.  Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery , 2015, Remote. Sens..

[74]  Jing Yuan,et al.  HyperDense-Net: A Hyper-Densely Connected CNN for Multi-Modal Image Segmentation , 2018, IEEE Transactions on Medical Imaging.

[75]  Cordelia Schmid,et al.  Computer Vision – ECCV 2012 , 2012, Lecture Notes in Computer Science.