Multilayer Feature Fusion Network for Scene Classification in Remote Sensing

The scene classification of high spatial resolution (HSR) images is a challenging task in the remote sensing community. How to construct a discriminative representation of the HSR scene is a key step to improve classification performance. In this letter, we propose a novel feature extraction method termed multilayer feature fusion network (MF2Net) for scene classification. At first, the transferred VGGNet-16 model is employed as a feature extractor to acquire multilayer convolutional features. Then, several layers including pooling, transformation, and fusion layers are designed to process hierarchical features in four branches, and the prediction probability can be obtained for classification. Finally, the proposed model is optimized by fine-tuning techniques, where a novel data augmentation approach is explored to improve generalization ability. As a result, MF2Net effectively applies useful information from multilayers to improve the accuracy of scene classification. The experimental results on AID and NWPU-RESISC45 data sets exhibit that the MF2Net method obtains quite competitive classification results compared with many state-of-the-art methods.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Xuelong Li,et al.  Scene Classification With Recurrent Attention of VHR Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Hong Huang,et al.  Combing Triple-Part Features of Convolutional Neural Networks for Scene Classification in Remote Sensing , 2019, Remote. Sens..

[4]  Cong Lin,et al.  Integrating Multilayer Features of Convolutional Neural Networks for Remote Sensing Scene Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Cong Lin,et al.  Automatic Updating of Land Cover Maps in Rapidly Urbanizing Regions by Relational Knowledge Transferring from GlobeLand30 , 2019, Remote. Sens..

[6]  Yanjiang Wang,et al.  Weighted Spatial Pyramid Matching Collaborative Representation for Remote-Sensing-Image Scene Classification , 2019, Remote. Sens..

[7]  Fahad Shahbaz Khan,et al.  Binary Patterns Encoded Convolutional Neural Networks for Texture Recognition and Remote Sensing Scene Classification , 2017, ArXiv.

[8]  Zhenfeng Shao,et al.  PatternNet: A Benchmark Dataset for Performance Evaluation of Remote Sensing Image Retrieval , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[9]  Hong Huo,et al.  Exploiting Convolutional Neural Networks With Deeply Local Description for Remote Sensing Image Classification , 2018, IEEE Access.

[10]  Xiangtao Zheng,et al.  A Deep Scene Representation for Aerial Scene Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Lei Guo,et al.  When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Qian Du,et al.  Fusing Local and Global Features for High-Resolution Scene Classification , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[13]  Hongxun Yao,et al.  Deep Feature Fusion for VHR Remote Sensing Scene Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Yanfei Zhong,et al.  Multi-class geospatial object detection based on a position-sensitive balancing framework for high spatial resolution remote sensing imagery , 2018 .

[15]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification: Benchmark and State of the Art , 2017, Proceedings of the IEEE.

[16]  Yunlong Yu,et al.  Aerial Scene Classification via Multilevel Fusion Based on Deep Convolutional Neural Networks , 2018, IEEE Geoscience and Remote Sensing Letters.

[17]  Yunlong Yu,et al.  A Two-Stream Deep Fusion Framework for High-Resolution Aerial Scene Classification , 2018, Comput. Intell. Neurosci..

[18]  Hong Huo,et al.  Global-Local Attention Network for Aerial Scene Classification , 2019, IEEE Access.

[19]  Wenzhong Guo,et al.  Land-Use Classification via Extreme Learning Classifier Based on Deep Convolutional Features , 2017, IEEE Geoscience and Remote Sensing Letters.

[20]  Xiao Xiang Zhu,et al.  Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources , 2017, IEEE Geoscience and Remote Sensing Magazine.

[21]  Yunlong Yu,et al.  Dense Connectivity Based Two-Stream Deep Feature Fusion Framework for Aerial Scene Classification , 2018, Remote. Sens..

[22]  Bo Du,et al.  Scene Classification via a Gradient Boosting Random Convolutional Network Framework , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Chao Huang,et al.  Scene Classification via Triplet Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[24]  Yishu Liu,et al.  Scene Classification Based on Two-Stage Deep Feature Fusion , 2018, IEEE Geoscience and Remote Sensing Letters.

[25]  Shutao Li,et al.  Remote Sensing Scene Classification Using Multilayer Stacked Covariance Pooling , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Antonio Plaza,et al.  Scale-Free Convolutional Neural Network for Remote Sensing Scene Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Gui-Song Xia,et al.  Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery , 2015, Remote. Sens..

[28]  Shiming Xiang,et al.  Aggregating Rich Hierarchical Features for Scene Classification in Remote Sensing Imagery , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[29]  Lei Guo,et al.  Remote Sensing Image Scene Classification Using Bag of Convolutional Features , 2017, IEEE Geoscience and Remote Sensing Letters.

[30]  Mohamed Farah,et al.  Remote Sensing Scene Classification Using Convolutional Features and Deep Forest Classifier , 2019, IEEE Geoscience and Remote Sensing Letters.

[31]  Guohui Tian,et al.  A novel scene classification model combining ResNet based transfer learning and data augmentation with a filter , 2019, Neurocomputing.

[32]  Ning Li,et al.  Multiscale deep features learning for land-use scene recognition , 2018 .

[33]  Gui-Song Xia,et al.  AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.