Fully Convolutional Neural Network with Augmented Atrous Spatial Pyramid Pool and Fully Connected Fusion Path for High Resolution Remote Sensing Image Segmentation

Recent developments in Convolutional Neural Networks (CNNs) have allowed for the achievement of solid advances in semantic segmentation of high-resolution remote sensing (HRRS) images. Nevertheless, the problems of poor classification of small objects and unclear boundaries caused by the characteristics of the HRRS image data have not been fully considered by previous works. To tackle these challenging problems, we propose an improved semantic segmentation neural network, which adopts dilated convolution, a fully connected (FC) fusion path and pre-trained encoder for the semantic segmentation task of HRRS imagery. The network is built with the computationally-efficient DeepLabv3 architecture, with added Augmented Atrous Spatial Pyramid Pool and FC Fusion Path layers. Dilated convolution enlarges the receptive field of feature points without decreasing the feature map resolution. The improved neural network architecture enhances HRRS image segmentation, reaching the classification accuracy of 91%, and the precision of recognition of small objects is improved. The applicability of the improved model to the remote sensing image segmentation task is verified.

[1]  Jiang-She Zhang,et al.  Big data analytics enabled by feature extraction based on partial independence , 2017, Neurocomputing.

[2]  Lu Yang,et al.  Semantic Segmentation for High Spatial Resolution Remote Sensing Images Based on Convolution Neural Network and Pyramid Pooling Module , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[3]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[5]  Rytis Maskeliunas,et al.  Reconstruction of 3D Object Shape Using Hybrid Modular Neural Network Architecture Trained on 3D Models from ShapeNetCore Dataset , 2019, Sensors.

[6]  Peter Reinartz,et al.  Aerial LaneNet: Lane-Marking Semantic Segmentation in Aerial Imagery Using Wavelet-Enhanced Cost-Sensitive Symmetric Fully Convolutional Neural Networks , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Xiangyu Zhang,et al.  Large Kernel Matters — Improve Semantic Segmentation by Global Convolutional Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Jun Rao,et al.  Road Segmentation Based on Hybrid Convolutional Network for High-Resolution Visible Remote Sensing Image , 2019, IEEE Geoscience and Remote Sensing Letters.

[9]  Pierre Alliez,et al.  Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Feng Li,et al.  Convolutional Neural Network-Based Remote Sensing Images Segmentation Method for Extracting Winter Wheat Spatial Distribution , 2018, Applied Sciences.

[11]  Gang Fu,et al.  Classification for High Resolution Remote Sensing Imagery Using a Fully Convolutional Network , 2017, Remote. Sens..

[12]  Jun Rao,et al.  A Y-Net deep learning method for road segmentation using high-resolution visible remote sensing images , 2019, Remote Sensing Letters.

[13]  Seyed Majid Azimi,et al.  Road Segmentation in SAR Satellite Images With Deep Fully Convolutional Neural Networks , 2018, IEEE Geoscience and Remote Sensing Letters.

[14]  Leonardo Vanneschi,et al.  Improved Fully Convolutional Network with Conditional Random Fields for Building Extraction , 2018, Remote. Sens..

[15]  Gregory Giuliani,et al.  Automated Classification of Terrestrial Images: The Contribution to the Remote Sensing of Snow Cover , 2019, Geosciences.

[16]  Peiyi Shen,et al.  Combined Energy Minimization for Image Reconstruction from Few Views , 2012 .

[17]  Bin Luo,et al.  Robust Autodual Morphological Profiles for the Classification of High-Resolution Satellite Images , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Daniel Buscombe,et al.  Landscape Classification with Deep Neural Networks , 2018, Geosciences.

[19]  Sabine Vanhuysse,et al.  Fully Convolutional Networks and Geographic Object-Based Image Analysis for the Classification of VHR Imagery , 2019, Remote. Sens..

[20]  Peiyun Hu,et al.  Finding Tiny Faces , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Menglong Yan,et al.  IoU-Adaptive Deformable R-CNN: Make Full Use of IoU for Multi-Class Object Detection in Remote Sensing Imagery , 2019, Remote. Sens..

[22]  Robertas Damasevicius,et al.  Multi-threaded learning control mechanism for neural networks , 2018, Future Gener. Comput. Syst..

[23]  Weiwei Sun,et al.  Fully Convolutional Networks for Semantic Segmentation of Very High Resolution Remotely Sensed Images Combined With DSM , 2018, IEEE Geoscience and Remote Sensing Letters.

[24]  Gaofeng Meng,et al.  SeNet: Structured Edge Network for Sea–Land Segmentation , 2017, IEEE Geoscience and Remote Sensing Letters.

[25]  Pengfei Chen,et al.  An Automatic Framework for Detecting and Characterizing Performance Degradation of Software Systems , 2014, IEEE Transactions on Reliability.

[26]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[27]  Wei Wei,et al.  Study on Remote Sensing Image Vegetation Classification Method Based on Decision Tree Classifier , 2018, 2018 IEEE Symposium Series on Computational Intelligence (SSCI).

[28]  Gang Yu,et al.  Learning a Discriminative Feature Network for Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[30]  Ying Wang,et al.  Gated Convolutional Neural Network for Semantic Segmentation in High-Resolution Images , 2017, Remote. Sens..

[31]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Xiao Xiang Zhu,et al.  Semantic segmentation of slums in satellite images using transfer learning on fully convolutional neural networks , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[33]  Ryosuke Shibasaki,et al.  Identification of Village Building via Google Earth Images and Supervised Machine Learning Methods , 2016, Remote. Sens..

[34]  Hai Wang,et al.  A Semantic Segmentation Algorithm Using FCN with Combination of BSLIC , 2018 .

[35]  Min Wang,et al.  Very high resolution remote sensing image classification with SEEDS-CNN and scale effect analysis for superpixel CNN classification , 2018, International Journal of Remote Sensing.

[36]  Ning Zhang,et al.  CoinNet: Copy Initialization Network for Multispectral Imagery Semantic Segmentation , 2019, IEEE Geoscience and Remote Sensing Letters.

[37]  Shunping Xiao,et al.  Deformable Faster R-CNN with Aggregating Multi-Layer Features for Partially Occluded Object Detection in Optical Remote Sensing Images , 2018, Remote. Sens..

[38]  Nuo Wang,et al.  Using satellite remote sensing and numerical modelling for the monitoring of suspended particulate matter concentration during reclamation construction at Dalian offshore airport in China , 2018 .

[39]  Xin Xu,et al.  Deformable ConvNet with Aspect Ratio Constrained NMS for Object Detection in Remote Sensing Imagery , 2017, Remote. Sens..

[40]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Peerapon Vateekul,et al.  Semantic Segmentation on Remotely Sensed Images Using an Enhanced Global Convolutional Network with Channel Attention and Domain Specific Transfer Learning , 2018, Remote. Sens..

[42]  Bertrand Le Saux,et al.  Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks , 2016, ACCV.

[43]  Wei Li,et al.  DeepUNet: A Deep Fully Convolutional Network for Pixel-Level Sea-Land Segmentation , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[44]  Jiangye Yuan,et al.  Automatic Building Extraction in Aerial Scenes Using Convolutional Networks , 2016, ArXiv.

[45]  Nataliia Kussul,et al.  Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data , 2017, IEEE Geoscience and Remote Sensing Letters.

[46]  Meng Lu,et al.  A scale robust convolutional neural network for automatic building extraction from aerial and satellite imagery , 2018, International Journal of Remote Sensing.