Convolutional Recurrent Network for Road Boundary Extraction

Creating high definition maps that contain precise information of static elements of the scene is of utmost importance for enabling self driving cars to drive safely. In this paper, we tackle the problem of drivable road boundary extraction from LiDAR and camera imagery. Towards this goal, we design a structured model where a fully convolutional network obtains deep features encoding the location and direction of road boundaries and then, a convolutional recurrent network outputs a polyline representation for each one of them. Importantly, our method is fully automatic and does not require a user in the loop. We showcase the effectiveness of our method on a large North American city where we obtain perfect topology of road boundaries 99.3% of the time at a high precision and recall.

[1]  Paul Newman,et al.  A variational approach to online road and path segmentation with monocular vision , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Sanja Fidler,et al.  Annotating Object Instances with a Polygon-RNN , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Eugenio Culurciello,et al.  LinkNet: Exploiting encoder representations for efficient semantic segmentation , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[4]  Sanja Fidler,et al.  Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++ , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Rahul Mohan,et al.  Deep Deconvolutional Networks for Scene Parsing , 2014, ArXiv.

[6]  Jean Ponce,et al.  General Road Detection From a Single Image , 2010, IEEE Transactions on Image Processing.

[7]  Sergio Casas,et al.  IntentNet: Learning to Predict Intention from Raw Sensor Data , 2018, CoRL.

[8]  Raquel Urtasun,et al.  Hierarchical Recurrent Attention Networks for Structured Online Maps , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[9]  Ivan Laptev,et al.  Automatic extraction of roads from aerial images based on scale space and snakes , 2000 .

[10]  Ethan Fetaya,et al.  StixelNet: A Deep Convolutional Network for Obstacle Detection and Road Segmentation , 2015, BMVC.

[11]  Min Bai,et al.  Deep Multi-Sensor Lane Detection , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[12]  Jan Dirk Wegner,et al.  A Higher-Order CRF Model for Road Network Extraction , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Sanja Fidler,et al.  Find your way by observing the sun and other semantic cues , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Julius Ziegler,et al.  StereoScan: Dense 3d reconstruction in real-time , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[15]  Min Bai,et al.  Learning Deep Structured Active Contours End-to-End , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Ivan Laptev,et al.  Multi-scale and Snakes for Automatic Road Extraction , 1998, ECCV.

[17]  S. Kammel,et al.  Lidar-based lane marker detection and mapping , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[18]  Jan-Michael Frahm,et al.  Detailed Real-Time Urban 3D Reconstruction from Video , 2007, International Journal of Computer Vision.

[19]  Josiane Zerubia,et al.  Higher Order Active Contours , 2006, International Journal of Computer Vision.

[20]  Raquel Urtasun,et al.  Learning to Localize Using a LiDAR Intensity Map , 2018, CoRL.

[21]  Costas Armenakis,et al.  Survey of Work on Road Extraction in Aerial and Satellite Images , 2002 .

[22]  Sanja Fidler,et al.  Enhancing Road Maps by Parsing Aerial Images Around the World , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[23]  Antonio M. López,et al.  Road Detection Based on Illuminant Invariance , 2011, IEEE Transactions on Intelligent Transportation Systems.

[24]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[25]  Konrad Schindler,et al.  Road networks as collections of minimum cost paths , 2015 .

[26]  Min Bai,et al.  TorontoCity: Seeing the World with a Million Eyes , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[27]  Stanislav S. Makhanov,et al.  A Family of Quadratic Snakes for Road Extraction , 2007, Asian Conference on Computer Vision.

[28]  Raquel Urtasun,et al.  DeepRoadMapper: Extracting Road Topology from Aerial Images , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Hans P. Moravec,et al.  High resolution maps from wide angle sonar , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[30]  Javier Civera,et al.  Real-time dense map fusion for stereo SLAM , 2018, Robotica.

[31]  Luc Van Gool,et al.  Iterative Deep Learning for Road Topology Extraction , 2018, BMVC.

[32]  Christian Heipke,et al.  Network snakes: graph-based object delineation with active contour models , 2010, Machine Vision and Applications.

[33]  Jan Dirk Wegner,et al.  Mind the Gap: Modeling Local and Global Context in (Road) Networks , 2014, GCPR.

[34]  D. Simonett,et al.  On the use of space photography for identifying transportation routes: A summary of problems , 1970 .

[35]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Geoffrey E. Hinton,et al.  Learning to Detect Roads in High-Resolution Aerial Images , 2010, ECCV.

[37]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[38]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[39]  Uwe Stilla,et al.  Classification With an Edge: Improving Semantic Image Segmentation with Boundary Detection , 2016, ISPRS Journal of Photogrammetry and Remote Sensing.

[40]  David J. DeWitt,et al.  RoadTracer: Automatic Extraction of Road Networks from Aerial Images , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Bin Yang,et al.  Deep Continuous Fusion for Multi-sensor 3D Object Detection , 2018, ECCV.

[42]  Raquel Urtasun,et al.  End-to-End Deep Structured Models for Drawing Crosswalks , 2018, ECCV.

[43]  Sanja Fidler,et al.  HD Maps: Fine-Grained Road Segmentation by Parsing Ground and Aerial Images , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Marc Pollefeys,et al.  Robust Dense Mapping for Large-Scale Dynamic Environments , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[45]  Ivan Laptev,et al.  Automatic extraction of roads from aerial images based on scale space and snakes , 2000, Machine Vision and Applications.

[46]  Franz Kummert,et al.  Spatial ray features for real-time ego-lane extraction , 2012, 2012 15th International IEEE Conference on Intelligent Transportation Systems.

[47]  Tommy Chang,et al.  Color model-based real-time learning for road following , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[48]  Uwe Stilla,et al.  SEMANTIC SEGMENTATION OF AERIAL IMAGES WITH AN ENSEMBLE OF CNNS , 2016 .

[49]  Sebastian Thrun,et al.  Adaptive Road Following using Self-Supervised Learning and Reverse Optical Flow , 2005, Robotics: Science and Systems.

[50]  Hsu-Yung Cheng,et al.  Lane Detection With Moving Vehicles in the Traffic Scenes , 2006, IEEE Transactions on Intelligent Transportation Systems.

[51]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[52]  Raquel Urtasun,et al.  Efficient Convolutions for Real-Time Semantic Segmentation of 3D Point Clouds , 2018, 2018 International Conference on 3D Vision (3DV).

[53]  Geoffrey E. Hinton,et al.  Learning to Label Aerial Images from Noisy Data , 2012, ICML.

[54]  Raquel Urtasun,et al.  Estimating Drivable Collision-Free Space from Monocular Video , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[55]  Daniel Cremers,et al.  B-Spline Modeling of Road Surfaces With an Application to Free-Space Estimation , 2009, IEEE Transactions on Intelligent Transportation Systems.

[56]  José A. Malpica,et al.  An automatic method for road extraction in rural and semi-urban areas starting from high resolution satellite imagery , 2005, Pattern Recognit. Lett..

[57]  Ruzena Bajcsy,et al.  Computer Recognition of Roads from Satellite Pictures , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[58]  John A. Richards,et al.  Remote Sensing Digital Image Analysis , 1986 .

[59]  Yann LeCun,et al.  Road Scene Segmentation from a Single Image , 2012, ECCV.