Remote sensing image scene classification using CNN-MLP with data augmentation

Abstract Classification of the very high-resolution (VHR) imagery scene has become a challenging problem. The convolutional neural network (CNN) has increased the accuracy in this area due to learning features. However, models based on CNN contain many deep layers for classifying images that are not perfect in describing the relationship between objects within the image. Therefore, an enhanced multilayer perceptron (MLP) depending on Adagrad optimizer is employed in the classification step in this paper as a deep classifier. Motivated by this idea, this paper proposes an effective classification model named CNN-MLP to utilize the benefits of these two techniques: CNN and MLP. The features are generated using pre-trained CNN without fully connected layers. Due to limited training images per class, the proposed model uses data augmentation techniques to expand the training images. Then, an MLP is used to classify the final feature maps into the specified classes. Three public remote sensing datasets of VHR images to evaluate the proposed CNN-MLP model: UC-Merced, Aerial Image (AID), and NWPU-RESISC45 datasets. The experiment's findings show that the proposed method will contribute to higher classification performance relative to state-of-the-art methods.

[1]  Gong Cheng,et al.  Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[2]  Fathi E. Abd El-Samie,et al.  Multispectral image compression with band ordering and wavelet transforms , 2013, Signal, Image and Video Processing.

[3]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Xiaopeng Fan,et al.  Lossy compression of satellite images with low impact on vegetation features , 2017, Multidimens. Syst. Signal Process..

[5]  Michael W. Marcellin,et al.  Regression Wavelet Analysis for Near-Lossless Remote Sensing Data Compression , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Luis Perez,et al.  The Effectiveness of Data Augmentation in Image Classification using Deep Learning , 2017, ArXiv.

[7]  Zhong Dong,et al.  Learning a robust CNN-based rotation insensitive model for ship detection in VHR remote sensing images , 2020 .

[8]  Xiangtao Zheng,et al.  Remote Sensing Scene Classification by Unsupervised Representation Learning , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Chao Yang,et al.  Concentric Circle Pooling in Deep Convolutional Networks for Remote Sensing Scene Classification , 2018, Remote. Sens..

[10]  Liangpei Zhang,et al.  A Deep-Local-Global Feature Fusion Framework for High Spatial Resolution Imagery Scene Classification , 2018, Remote. Sens..

[11]  Lorenzo Bruzzone,et al.  Remote Sensing Image Scene Classification with Deep Neural Networks in JPEG 2000 Compressed Domain , 2020, ArXiv.

[12]  Lei Guo,et al.  Remote Sensing Image Scene Classification Using Bag of Convolutional Features , 2017, IEEE Geoscience and Remote Sensing Letters.

[13]  Jon Atli Benediktsson,et al.  Object-Oriented Key Point Vector Distance for Binary Land Cover Change Detection Using VHR Remote Sensing Images , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Antonio Plaza,et al.  Skip-Connected Covariance Network for Remote Sensing Scene Classification , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Huan Xie,et al.  Multi-view feature learning for VHR remote sensing image classification , 2020, Multimedia Tools and Applications.

[16]  Jefersson Alex dos Santos,et al.  Towards better exploiting convolutional neural networks for remote sensing scene classification , 2016, Pattern Recognit..

[17]  Yanfei Liu,et al.  Scene Classification Based on a Deep Random-Scale Stretched Convolutional Neural Network , 2018, Remote. Sens..

[18]  Fahad Shahbaz Khan,et al.  Binary Patterns Encoded Convolutional Neural Networks for Texture Recognition and Remote Sensing Scene Classification , 2017, ArXiv.

[19]  Dazhuan Xu,et al.  A Fast Deep Perception Network for Remote Sensing Scene Classification , 2020, Remote. Sens..

[20]  Haitao Guo,et al.  An Efficient and Lightweight Convolutional Neural Network for Remote Sensing Image Scene Classification , 2020, Sensors.

[21]  Gui-Song Xia,et al.  Extreme value theory-based calibration for the fusion of multiple features in high-resolution satellite scene classification , 2013 .

[22]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[23]  Ling Shao,et al.  Multi-Granularity Canonical Appearance Pooling for Remote Sensing Scene Classification , 2020, IEEE Transactions on Image Processing.

[24]  Dan Zeng,et al.  Improving Remote Sensing Scene Classification by Integrating Global-Context and Local-Object Features , 2018, Remote. Sens..

[25]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification Using Rearranged Local Features , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Sami D. Alaruri Multiwavelength laser induced fluorescence (LIF) LIDAR system for remote detection and identification of oil spills , 2019, Optik.

[27]  Jiancheng Luo,et al.  Geo-parcel-based crop classification in very-high-resolution images via hierarchical perception , 2020 .

[28]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[29]  Yanjiang Wang,et al.  Weighted Spatial Pyramid Matching Collaborative Representation for Remote-Sensing-Image Scene Classification , 2019, Remote. Sens..

[30]  Lijun Zhao,et al.  Remote Sensing Image Scene Classification Using CNN-CapsNet , 2019, Remote. Sens..

[31]  Shun Zhang,et al.  Rotation-Invariant Feature Learning for Object Detection in VHR Optical Remote Sensing Images by Double-Net , 2020, IEEE Access.

[32]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification: Benchmark and State of the Art , 2017, Proceedings of the IEEE.

[33]  Guangyao Shi,et al.  Multilayer Feature Fusion Network for Scene Classification in Remote Sensing , 2020, IEEE Geoscience and Remote Sensing Letters.

[34]  Wei Xiong,et al.  An End-to-End Local-Global-Fusion Feature Extraction Network for Remote Sensing Image Scene Classification , 2019, Remote. Sens..

[35]  Liangpei Zhang,et al.  Scene Classification Based on the Fully Sparse Semantic Topic Model , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Yunlong Yu,et al.  A Two-Stream Deep Fusion Framework for High-Resolution Aerial Scene Classification , 2018, Comput. Intell. Neurosci..

[37]  Xiangtao Zheng,et al.  A Deep Scene Representation for Aerial Scene Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Feng Yang,et al.  Multi-Scale Feature Integrated Attention-Based Rotation Network for Object Detection in VHR Aerial Images , 2020, Sensors.

[39]  Qian Du,et al.  Fusing Local and Global Features for High-Resolution Scene Classification , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[40]  Hongxun Yao,et al.  Deep Feature Fusion for VHR Remote Sensing Scene Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Fathi E. Abd El-Samie,et al.  Satellite multispectral image compression based on removing sub-bands , 2017 .

[42]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[43]  Yuhuan Ren,et al.  Improved CNN Classification Method for Groups of Buildings Damaged by Earthquake, Based on High Resolution Remote Sensing Images , 2020, Remote. Sens..

[44]  Xiangtao Zheng,et al.  Remote Sensing Scene Classification by Gated Bidirectional Network , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[45]  Varun Tiwari,et al.  Efficient FPGA Implementation of Multilayer Perceptron for Real-Time Human Activity Classification , 2019, IEEE Access.

[46]  Xuelong Li,et al.  Feature Sparsity in Convolutional Neural Networks for Scene Classification of Remote Sensing Image , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[47]  Hong Huang,et al.  Combing Triple-Part Features of Convolutional Neural Networks for Scene Classification in Remote Sensing , 2019, Remote. Sens..

[48]  Tatsuya Harada,et al.  Between-Class Learning for Image Classification , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[49]  Lei Guo,et al.  When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs , 2018, IEEE Transactions on Geoscience and Remote Sensing.