Deep Learning Approaches Applied to Remote Sensing Datasets for Road Extraction: A State-Of-The-Art Review

One of the most challenging research subjects in remote sensing is feature extraction, such as road features, from remote sensing images. Such an extraction influences multiple scenes, including map updating, traffic management, emergency tasks, road monitoring, and others. Therefore, a systematic review of deep learning techniques applied to common remote sensing benchmarks for road extraction is conducted in this study. The research is conducted based on four main types of deep learning methods, namely, the GANs model, deconvolutional networks, FCNs, and patch-based CNNs models. We also compare these various deep learning models applied to remote sensing datasets to show which method performs well in extracting road parts from high-resolution remote sensing images. Moreover, we describe future research directions and research gaps. Results indicate that the largest reported performance record is related to the deconvolutional nets applied to remote sensing images, and the F1 score metric of the generative adversarial network model, DenseNet method, and FCN-32 applied to UAV and Google Earth images are high: 96.08%, 95.72%, and 94.59%, respectively.

[1]  Biswajeet Pradhan,et al.  Urban Development Modeling Using Integrated Fuzzy Systems, Ordered Weighted Averaging (OWA), and Geospatial Techniques , 2020, Sustainability.

[2]  Yang Wang,et al.  Road Extraction From High-Resolution Satellite Images Based on Multiple Descriptors , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[3]  Fang Miao,et al.  HsgNet: A Road Extraction Network Based on Global Perception of High-Order Spatial Information , 2019, ISPRS Int. J. Geo Inf..

[4]  Xinchang Zhang,et al.  Road Extraction of High-Resolution Remote Sensing Images Derived from DenseUNet , 2019, Remote. Sens..

[5]  Biswajeet Pradhan,et al.  Extraction of road features from UAV images using a novel level set segmentation approach , 2019, International Journal of Urban Sciences.

[6]  Yang Chen,et al.  Road Extraction from High-Resolution Remote Sensing Imagery Using Refined Deep Residual Convolutional Neural Network , 2019, Remote. Sens..

[7]  Pengfei Xu,et al.  Multiscale road centerlines extraction from high-resolution aerial imagery , 2019, Neurocomputing.

[8]  Jun Rao,et al.  A Y-Net deep learning method for road segmentation using high-resolution visible remote sensing images , 2019, Remote Sensing Letters.

[9]  Hao He,et al.  Road segmentation of cross-modal remote sensing images using deep segmentation network and transfer learning , 2019, Ind. Robot.

[10]  Neelanshi Varia,et al.  DeepExt: A Convolution Neural Network for Road Extraction using RGB images captured by UAV , 2018, 2018 IEEE Symposium Series on Computational Intelligence (SSCI).

[11]  Jian-Jiun Ding,et al.  Accurate Road Detection from Satellite Images Using Modified U-net , 2018, 2018 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS).

[12]  Yongyang Xu,et al.  Road Extraction from High-Resolution Remote Sensing Imagery Using Deep Learning , 2018, Remote. Sens..

[13]  Alexey Shvets,et al.  Fully Convolutional Network for Automatic Road Extraction from Satellite Imagery , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  Yongyang Xu,et al.  A Research on Extracting Road Network from High Resolution Remote Sensing Imagery , 2018, 2018 26th International Conference on Geoinformatics.

[15]  Jigar Doshi,et al.  Residual Inception Skip Network for Binary Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[16]  Ming Wu,et al.  D-LinkNet: LinkNet with Pretrained Encoder and Dilated Convolution for High Resolution Satellite Imagery Road Extraction , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[17]  Wei Xia,et al.  Road Extraction from High Resolution Image with Deep Convolution Network – A Case Study of GF-2 Image , 2018 .

[18]  Mojgan Pashaie Nejad,et al.  Investigation of SVM and Level Set Interactive Methods for Road Extraction from Google Earth Images , 2018, Journal of the Indian Society of Remote Sensing.

[19]  Rameen Abdal,et al.  UFCN: a fully convolutional neural network for road extraction in RGB imagery acquired by remote sensing from an unmanned aerial vehicle , 2018 .

[20]  Seyed Majid Azimi,et al.  Road Segmentation in SAR Satellite Images With Deep Fully Convolutional Neural Networks , 2018, IEEE Geoscience and Remote Sensing Letters.

[21]  Qingjie Liu,et al.  Road Extraction by Deep Residual U-Net , 2017, IEEE Geoscience and Remote Sensing Letters.

[22]  Emil Slusanschi,et al.  Creating Roadmaps in Aerial Images with Generative Adversarial Networks and Smoothing-Based Optimization , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[23]  Wei Lee Woon,et al.  Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks , 2017 .

[24]  Peerapon Vateekul,et al.  An Enhanced Deep Convolutional Encoder-Decoder Network for Road Segmentation on Aerial Imagery , 2017, International Conference on Computer and Information Technology.

[25]  Peerapon Vateekul,et al.  Road Segmentation of Remotely-Sensed Images Using Deep Convolutional Neural Networks with Landscape Metrics and Conditional Random Fields , 2017, Remote. Sens..

[26]  Yongyang Xu,et al.  Quality assessment of building footprint data using a deep autoencoder network , 2017, Int. J. Geogr. Inf. Sci..

[27]  Abolfazl Abdollahi,et al.  Semi automatic road extraction from digital images , 2017 .

[28]  Bertrand Le Saux,et al.  Segment-before-Detect: Vehicle Detection and Classification through Semantic Segmentation of Aerial Images , 2017, Remote. Sens..

[29]  Prashanth Reddy Marpu,et al.  Hierarchical graph-based segmentation for extracting road networks from high-resolution satellite images , 2017 .

[30]  Zulin Wang,et al.  Road Structure Refined CNN for Road Extraction in Aerial Image , 2017, IEEE Geoscience and Remote Sensing Letters.

[31]  Qi Tian,et al.  Road Recognition From Remote Sensing Imagery Using Incremental Learning , 2017, IEEE Transactions on Intelligent Transportation Systems.

[32]  Shiming Xiang,et al.  Automatic Road Detection and Centerline Extraction via Cascaded End-to-End Convolutional Neural Network , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[33]  Pierre Alliez,et al.  Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Qian Du,et al.  Hyperspectral Image Classification Using Deep Pixel-Pair Features , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Georgios Tziritas,et al.  MRF-based segmentation and unsupervised classification for building and road detection in peri-urban areas of high-resolution satellite images , 2016 .

[36]  Camille Couprie,et al.  Semantic Segmentation using Adversarial Networks , 2016, NIPS 2016.

[37]  Michele Volpi,et al.  Dense Semantic Labeling of Subdecimeter Resolution Images With Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Jianhua Wang,et al.  A New Approach to Urban Road Extraction Using High-Resolution Aerial Image , 2016, ISPRS Int. J. Geo Inf..

[39]  Yao Yu,et al.  Road network extraction via deep learning and line integral convolution , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[40]  Han Jiang,et al.  Fully convolutional networks for building and road extraction: Preliminary results , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[41]  Pierre Alliez,et al.  Fully convolutional neural networks for remote sensing image classification , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[42]  Nan Yang,et al.  A review of road extraction from remote sensing images , 2016 .

[43]  J. Senthilnath,et al.  Detection of tomatoes using spectral-spatial methods in remotely sensed RGB images captured by UAV , 2016 .

[44]  B. Pradhan,et al.  Analysis on causes of flash flood in Jeddah city (Kingdom of Saudi Arabia) of 2009 and 2011 using multi-sensor remote sensing data and GIS , 2016 .

[45]  Richard Lepage,et al.  Road Extraction From Very High Resolution Remote Sensing Optical Images Based on Texture Analysis and Beamlet Transform , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[46]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[47]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Sanja Fidler,et al.  Enhancing Road Maps by Parsing Aerial Images Around the World , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[49]  Gui-Song Xia,et al.  Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery , 2015, Remote. Sens..

[50]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Huayi Wu,et al.  Main Road Extraction from ZY-3 Grayscale Imagery Based on Directional Mathematical Morphology and VGI Prior Knowledge in Urban Areas , 2015, PloS one.

[52]  Paolo Gamba,et al.  An Object-Based Method for Road Network Extraction in VHR Satellite Images , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[53]  Jun Wang,et al.  Road network extraction: a neural-dynamic framework based on deep learning and a finite state machine , 2015 .

[54]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[55]  Biswajeet Pradhan,et al.  GIS-based sustainable city compactness assessment using integration of MCDM, Bayes theorem and RADAR technology , 2015 .

[56]  Bisheng Yang,et al.  Automatic registration of UAV-borne sequent images and LiDAR data , 2015 .

[57]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[58]  Trevor Darrell,et al.  Fully convolutional networks for semantic segmentation , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[60]  Aaron C. Courville,et al.  Generative Adversarial Networks , 2014, 1406.2661.

[61]  Wenzhong Shi,et al.  An Integrated Method for Urban Main-Road Centerline Extraction From Optical Remotely Sensed Imagery , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[62]  Farid Melgani,et al.  Detecting Cars in UAV Images With a Catalog-Based Approach , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[63]  Jan Dirk Wegner,et al.  A Higher-Order CRF Model for Road Network Extraction , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[64]  Jayan Sri Jeewantha Wijesingha,et al.  AUTOMATIC ROAD FEATURE EXTRACTION FROM HIGH RESOLUTION SATELLITE IMAGES USING LVQ NEURAL NETWORKS , 2013 .

[65]  Jixiang Sun,et al.  Road Extraction from High-Resolution SAR Images via Automatic Local Detecting and Human-Guided Global Tracking , 2012 .

[66]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[67]  Mingsheng Liao,et al.  Road Extraction From SAR Imagery Based on Multiscale Geometric Analysis of Detector Responses , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[68]  Qihao Weng,et al.  Remote sensing of impervious surfaces in the urban areas: Requirements, methods, and trends , 2012 .

[69]  Rohit Maurya,et al.  Road extraction using K-Means clustering and morphological operations , 2011, 2011 International Conference on Image Information Processing.

[70]  Geoffrey E. Hinton,et al.  Learning to Detect Roads in High-Resolution Aerial Images , 2010, ECCV.

[71]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement , 2009, BMJ : British Medical Journal.

[72]  D. Moher,et al.  Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement , 2009, BMJ : British Medical Journal.

[73]  Wenzhong Shi,et al.  The recognition of road network from high‐resolution satellite remotely sensed data using image morphological characteristics , 2005 .

[74]  Albert Jones,et al.  Toward modeling and simulation of critical national infrastructure interdependencies , 2005 .

[75]  José A. Malpica,et al.  An automatic method for road extraction in rural and semi-urban areas starting from high resolution satellite imagery , 2005, Pattern Recognit. Lett..

[76]  Xia Li,et al.  Road Detection From Remote Sensing Images by Generative Adversarial Networks , 2018, IEEE Access.

[77]  Tingting Lu,et al.  Road Extraction From a High Spatial Resolution Remote Sensing Image Based on Richer Convolutional Features , 2018, IEEE Access.

[78]  Chandran Saravanan,et al.  Automated Road Extraction From High Resolution Satellite Images , 2016 .

[79]  Takayoshi Yamashita,et al.  Multiple Object Extraction from Aerial Imagery with Convolutional Neural Networks , 2016, IRIACV.

[80]  Igor Sevo,et al.  Convolutional Neural Network Based Automatic Object Detection on Aerial Images , 2016, IEEE Geoscience and Remote Sensing Letters.

[81]  Shijian Tang,et al.  Object Detection based on Convolutional Neural Network , 2015 .

[82]  K. Moffett,et al.  Remote Sens , 2015 .

[83]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.