DV3+HED+: a DCNN-based framework to monitor temporary works and ESAs in railway construction project using VHR satellite images

Abstract. Current very high-resolution (VHR) satellite images enable the detailed monitoring of the Earth and can capture the ongoing works of railway construction. We presented an integrated framework applied to monitoring railway construction in China using QuickBird, Gaofen-2, and Google Earth VHR satellite images. We also constructed a deep convolutional neural network-based semantic segmentation network to label the temporary works, such as borrow and spoil areas, camp, beam yard, and environmentally sensitive areas such as resident houses throughout the whole railway construction project using VHR satellite images. In addition, we employed the holistically nested edge detection subnetwork to refine the boundary details and a cross entropy loss function to fit the sample class disequilibrium problem. Our semantic segmentation network was trained on 471, validated on 58, and tested on the 58 VHR true color images. The experiment results showed that compared with the existing state-of-the-art DeepLabV3 plus approach, our approach has improvements with an overall accuracy of more than 80%.

[1]  Pierre Alliez,et al.  High-Resolution Semantic Labeling with Convolutional Neural Networks , 2016 .

[2]  Tomas Maul,et al.  Satellite remote-sensing monitoring of a railway construction project , 2018 .

[3]  Ian D. Reid,et al.  RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[6]  Thomas Blaschke,et al.  Object based image analysis for remote sensing , 2010 .

[7]  Jefersson Alex dos Santos,et al.  Do deep features generalize from everyday objects to remote sensing and aerial scenes domains? , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[9]  Mostafa Arastounia,et al.  Automated Recognition of Railroad Infrastructure in Rural Areas from LIDAR Data , 2015, Remote. Sens..

[10]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[11]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[12]  Xiaocong Xu,et al.  Building Footprint Extraction from High-Resolution Images via Spatial Residual Inception Convolutional Neural Network , 2019, Remote. Sens..

[13]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[15]  Gang Yu,et al.  BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation , 2018, ECCV.

[16]  Xiaohui Liang,et al.  DOOBNet: Deep Object Occlusion Boundary Detection from an Image , 2018, ACCV.

[17]  Jun Rao,et al.  A Y-Net deep learning method for road segmentation using high-resolution visible remote sensing images , 2019, Remote Sensing Letters.

[18]  Xiaolin Du,et al.  Monitoring abandoned dreg fields of high-speed railway construction with UAV remote sensing technology , 2015, Intelligent Earth Observing Systems.

[19]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[20]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Jungho Im,et al.  Support vector machines in remote sensing: A review , 2011 .

[22]  Ramon F. Hanssen,et al.  Nationwide Railway Monitoring Using Satellite SAR Interferometry , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[23]  Chao Tian,et al.  Dense Fusion Classmate Network for Land Cover Classification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[24]  Yu Liu,et al.  Hourglass-ShapeNetwork Based Semantic Segmentation for High Resolution Aerial Imagery , 2017, Remote. Sens..

[25]  Michele Volpi,et al.  Dense Semantic Labeling of Subdecimeter Resolution Images With Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Eugenio Culurciello,et al.  LinkNet: Exploiting encoder representations for efficient semantic segmentation , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[27]  Mahesh Pal,et al.  Random forest classifier for remote sensing classification , 2005 .

[28]  Uwe Stilla,et al.  Classification With an Edge: Improving Semantic Image Segmentation with Boundary Detection , 2016, ISPRS Journal of Photogrammetry and Remote Sensing.

[29]  Edit J. Kaminsky,et al.  Neural network classification of remote-sensing data , 1995 .

[30]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[31]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[33]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[35]  Jianbo Liu,et al.  Pixel-Wise Classification Method for High Resolution Remote Sensing Imagery Using Deep Neural Networks , 2018, ISPRS Int. J. Geo Inf..

[36]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Alessandro Ferretti,et al.  Application of satellite radar interferometry for tunnel and underground infrastructures damage assessment and monitoring , 2013 .