Symmetrical Dense-Shortcut Deep Fully Convolutional Networks for Semantic Segmentation of Very-High-Resolution Remote Sensing Images

Semantic segmentation has emerged as a mainstream method in very-high-resolution remote sensing land-use/land-cover applications. In this paper, we first review the state-of-the-art semantic segmentation models in both computer vision and remote sensing fields. Subsequently, we introduce two semantic segmentation frameworks: SNFCN and SDFCN, both of which contain deep fully convolutional networks with shortcut blocks. We adopt an overlay strategy as the postprocessing method. Based on our frameworks, we conducted experiments on two online ISPRS datasets: Vaihingen and Potsdam. The results indicate that our frameworks achieve higher overall accuracy than the classic FCN-8s and SegNet models. In addition, our postprocessing method can increase the overall accuracy by about 1%–2% and help to eliminate “salt and pepper” phenomena and block effects.

[1]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[2]  William J. Emery,et al.  Object-Based Convolutional Neural Network for High-Resolution Imagery Classification , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[3]  Jamie Sherrah,et al.  Fully Convolutional Networks for Dense Semantic Labelling of High-Resolution Aerial Imagery , 2016, ArXiv.

[4]  Jie Shan,et al.  Object-based urban land cover classification using rule inheritance over very high-resolution multisensor and multitemporal data , 2016 .

[5]  한보형,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015 .

[6]  Vibhav Vineet,et al.  Conditional Random Fields as Recurrent Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[7]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  V. Liesenberg,et al.  Object-oriented and pixel-based classification approaches to classify tropical successional stages using airborne high–spatial resolution images , 2016 .

[9]  Liangpei Zhang,et al.  An Efficient and Robust Integrated Geospatial Object Detection Framework for High Spatial Resolution Remote Sensing Imagery , 2017, Remote. Sens..

[10]  Brian McWilliams,et al.  The Shattered Gradients Problem: If resnets are the answer, then what is the question? , 2017, ICML.

[11]  Gregory J. McDermid,et al.  Object-based approaches to change analysis and thematic map update: challenges and limitations , 2008 .

[12]  Indra Jaya,et al.  Object-based Image Analysis for Coral Reef Benthic Habitat Mapping with Several Classification Algorithms , 2015 .

[13]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[14]  Jamie Sherrah,et al.  Effective semantic pixel labelling with convolutional networks and Conditional Random Fields , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[15]  Bertrand Le Saux,et al.  Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks , 2016, ACCV.

[16]  Bo Du,et al.  Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art , 2016, IEEE Geoscience and Remote Sensing Magazine.

[17]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[18]  S D Walter,et al.  A reappraisal of the kappa coefficient. , 1988, Journal of clinical epidemiology.

[19]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Yuji Murayama,et al.  Pixel-based and object-based classifications using high- and medium-spatial-resolution imageries in the urban and suburban landscapes , 2015 .

[21]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[22]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Geoffrey J. Hay,et al.  Object-based Image Analysis : Strengths , Weaknesses , Opportunities and Threats ( Swot ) , 2006 .

[24]  Xiuping Jia,et al.  Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Thomas Blaschke,et al.  Geographic Object-Based Image Analysis – Towards a new paradigm , 2014, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[26]  Qing Wang,et al.  Object-Based Land-Cover Supervised Classification for Very-High-Resolution UAV Images Using Stacked Denoising Autoencoders , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[27]  Michael Kampffmeyer,et al.  Semantic Segmentation of Small Objects and Modeling of Uncertainty in Urban Remote Sensing Images Using Deep Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[28]  Fan Zhang,et al.  Deep Convolutional Neural Networks for Hyperspectral Image Classification , 2015, J. Sensors.

[29]  Geoffrey J. Hay,et al.  Geographic Object-Based Image Analysis (GEOBIA): A new name for a new discipline , 2008 .

[30]  Geoffrey E. Hinton,et al.  Deep Boltzmann Machines , 2009, AISTATS.

[31]  Pierre Alliez,et al.  Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Weiqi Zhou,et al.  Comparing Machine Learning Classifiers for Object-Based Land Cover Classification Using Very High Resolution Imagery , 2014, Remote Sensing.

[33]  Uwe Stilla,et al.  SEMANTIC SEGMENTATION OF AERIAL IMAGES WITH AN ENSEMBLE OF CNNS , 2016 .

[34]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Dongmei Chen,et al.  Change detection from remotely sensed images: From pixel-based to object-based approaches , 2013 .

[36]  Markus Gerke,et al.  The ISPRS benchmark on urban object classification and 3D building reconstruction , 2012 .

[37]  Thomas Blaschke,et al.  Object based image analysis for remote sensing , 2010 .

[38]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[39]  Uwe Stilla,et al.  Classification With an Edge: Improving Semantic Image Segmentation with Boundary Detection , 2016, ISPRS Journal of Photogrammetry and Remote Sensing.

[40]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[41]  L. Penrose.  The Elementary Statistics of Majority Voting , 1946 .

[42]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[43]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[44]  Steven E. Franklin,et al.  A comparison of pixel-based and object-based image analysis with selected machine learning algorithms for the classification of agricultural landscapes using SPOT-5 HRG imagery , 2012 .

[45]  Geoffrey E. Hinton,et al.  Machine Learning for Aerial Image Labeling , 2013 .

[46]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Han Jiang,et al.  Fully convolutional networks for building and road extraction: Preliminary results , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[48]  Guosheng Lin,et al.  Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[50]  Carlo Gatta,et al.  Unsupervised Deep Feature Extraction for Remote Sensing Image Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[51]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[52]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[53]  Mi Zhang,et al.  Learning Dual Multi-Scale Manifold Ranking for Semantic Segmentation of High-Resolution Images , 2017, Remote. Sens..

[54]  Ting Liu,et al.  Recent advances in convolutional neural networks , 2015, Pattern Recognit..

[55]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[56]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[57]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[58]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[59]  Josef Strobl,et al.  What’s wrong with pixels? Some recent developments interfacing remote sensing and GIS , 2001 .

[60]  L. Durieux,et al.  Advances in Geographic Object-Based Image Analysis with ontologies: A review of main contributions and limitations from a remote sensing perspective , 2013 .