Automatic Road Detection and Centerline Extraction via Cascaded End-to-End Convolutional Neural Network

Accurate road detection and centerline extraction from very high resolution (VHR) remote sensing imagery are of central importance in a wide range of applications. Due to the complex backgrounds and occlusions of trees and cars, most road detection methods bring in the heterogeneous segments; besides for the centerline extraction task, most current approaches fail to extract a wonderful centerline network that appears smooth, complete, as well as single-pixel width. To address the above-mentioned complex issues, we propose a novel deep model, i.e., a cascaded end-to-end convolutional neural network (CasNet), to simultaneously cope with the road detection and centerline extraction tasks. Specifically, CasNet consists of two networks. One aims at the road detection task, whose strong representation ability is well able to tackle the complex backgrounds and occlusions of trees and cars. The other is cascaded to the former one, making full use of the feature maps produced formerly, to obtain the good centerline extraction. Finally, a thinning algorithm is proposed to obtain smooth, complete, and single-pixel width road centerline network. Extensive experiments demonstrate that CasNet outperforms the state-of-the-art methods greatly in learning quality and learning speed. That is, CasNet exceeds the comparing methods by a large margin in quantitative performance, and it is nearly 25 times faster than the comparing methods. Moreover, as another contribution, a large and challenging road centerline data set for the VHR remote sensing image will be publicly available for further studies.

[1]  Gui-Song Xia,et al.  Deep Learning for Remote Sensing Image Understanding , 2016, J. Sensors.

[2]  Sukhendu Das,et al.  Use of Salient Features for the Design of a Multistage Framework to Extract Roads From High-Resolution Multispectral Satellite Images , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[4]  William M. Wells,et al.  Medical Image Computing and Computer-Assisted Intervention — MICCAI’98 , 1998, Lecture Notes in Computer Science.

[5]  Xin Huang,et al.  Road centreline extraction from high‐resolution imagery based on multiscale structural features and support vector machines , 2009 .

[6]  Wenzhong Shi,et al.  Spectral–Spatial Classification and Shape Features for Urban Road Centerline Extraction , 2014, IEEE Geoscience and Remote Sensing Letters.

[7]  Gui-Song Xia,et al.  Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery , 2015, Remote. Sens..

[8]  Ying Wang,et al.  Accurate urban road centerline extraction from VHR imagery via multiscale segmentation and tensor voting , 2015, Neurocomputing.

[9]  Jan Dirk Wegner,et al.  A Higher-Order CRF Model for Road Network Extraction , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Bo Du,et al.  Weakly Supervised Learning Based on Coupled Convolutional Neural Networks for Aircraft Detection , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Qiaoping Zhang,et al.  Benefit of the angular texture signature for the separation of parking lots and roads on high resolution multi-spectral imagery , 2006, Pattern Recognit. Lett..

[12]  B. Wessel,et al.  ANALYSIS OF AUTOMATIC ROAD EXTRACTION RESULTS FROM AIRBORNE SAR IMAGERY , 2003 .

[13]  Jacky Desachy,et al.  Geographic information system updating using remote sensing images , 2002, Pattern Recognit. Lett..

[14]  Liangpei Zhang,et al.  Scene Classification Based on the Multifeature Fusion Probabilistic Topic Model for High Spatial Resolution Remote Sensing Imagery , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[16]  Jefersson Alex dos Santos,et al.  Towards better exploiting convolutional neural networks for remote sensing scene classification , 2016, Pattern Recognit..

[17]  Cem Ünsalan,et al.  Road Network Detection Using Probabilistic and Graph Theoretical Methods , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[19]  Jian Sun,et al.  Instance-Aware Semantic Segmentation via Multi-task Network Cascades , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Min Wang,et al.  Segmentation of High Spatial Resolution Remote Sensing Imagery Based on Hard-Boundary Constraint and Two-Stage Merging , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Graham W. Taylor,et al.  Deconvolutional networks , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Qingquan Li,et al.  A Sensor-Fusion Drivable-Region and Lane-Detection System for Autonomous Vehicle Navigation in Challenging Road Scenarios , 2014, IEEE Transactions on Vehicular Technology.

[23]  Yongjun Zhang,et al.  Road Centerline Extraction in Complex Urban Scenes From LiDAR Data Based on Multiple Features , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[24]  Shiming Xiang,et al.  Road Centerline Extraction via Semisupervised Segmentation and Multidirection Nonmaximum Suppression , 2016, IEEE Geoscience and Remote Sensing Letters.

[25]  Wenzhong Shi,et al.  An Integrated Method for Urban Main-Road Centerline Extraction From Optical Remotely Sensed Imagery , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Yoko NISHIMURA,et al.  Google Earth , 2008, Encyclopedia of GIS.

[27]  Vincent Lepetit,et al.  Projection onto the Manifold of Elongated Structures for Accurate Extraction , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[28]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[29]  Christian Heipke,et al.  EVALUATION OF AUTOMATIC ROAD EXTRACTION , 2007 .

[30]  Wenzhong Shi,et al.  The recognition of road network from high‐resolution satellite remotely sensed data using image morphological characteristics , 2005 .

[31]  Atinderpreet Kaur,et al.  Various Methods of Road Extraction from Satellite Images A Review , 2015 .

[32]  Ashok Samal,et al.  Semi-Automated Road Detection From High Resolution Satellite Images by Directional Morphological Enhancement and Segmentation Techniques , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[33]  Liangpei Zhang,et al.  Scene semantic classification based on random-scale stretched convolutional neural network for high-spatial resolution remote sensing imagery , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[34]  Lin Yan,et al.  LEGION-Based Automatic Road Extraction From Satellite Imagery , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[36]  Juan B. Mena,et al.  State of the art on automatic road extraction for GIS update: a novel classification , 2003, Pattern Recognit. Lett..

[37]  Haipeng Wang,et al.  Target Classification Using the Deep Convolutional Networks for SAR Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[38]  D. Civco,et al.  Road Extraction Using SVM and Image Segmentation , 2004 .

[39]  Terry M. Peters,et al.  Medical Image Computing and Computer Assisted Intervention – MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part III , 2019, MICCAI.

[40]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[41]  Yann LeCun,et al.  Road Scene Segmentation from a Single Image , 2012, ECCV.

[42]  Wenzhong Shi,et al.  Road Centerline Extraction From High-Resolution Imagery Based on Shape Features and Multivariate Adaptive Regression Splines , 2013, IEEE Geoscience and Remote Sensing Letters.

[43]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Costas Armenakis,et al.  Survey of Work on Road Extraction in Aerial and Satellite Images , 2002 .

[45]  Paolo Gamba,et al.  Improving urban road extraction in high-resolution images exploiting directional filtering, perceptual grouping, and simple topological concepts , 2006, IEEE Geoscience and Remote Sensing Letters.

[46]  Vincent Lepetit,et al.  Multiscale Centerline Detection by Learning a Scale-Space Distance Transform , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[48]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[50]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[51]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[52]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[53]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[54]  Geoffrey E. Hinton,et al.  Learning to Detect Roads in High-Resolution Aerial Images , 2010, ECCV.

[55]  Bo Du,et al.  Scene Classification via a Gradient Boosting Random Convolutional Network Framework , 2016, IEEE Transactions on Geoscience and Remote Sensing.