End-to-End Deep Structured Models for Drawing Crosswalks

In this paper we address the problem of detecting crosswalks from LiDAR and camera imagery. Towards this goal, given multiple LiDAR sweeps and the corresponding imagery, we project both inputs onto the ground surface to produce a top down view of the scene. We then leverage convolutional neural networks to extract semantic cues about the location of the crosswalks. These are then used in combination with road centerlines from freely available maps (e.g., OpenStreetMaps) to solve a structured optimization problem which draws the final crosswalk boundaries. Our experiments over crosswalks in a large city area show that 96.6% automation can be achieved.

[1]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[2]  한보형,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015 .

[3]  Vibhav Vineet,et al.  Conditional Random Fields as Recurrent Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[4]  Philip H. S. Torr,et al.  Automatic dense visual semantic mapping from street-level imagery , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[6]  Ronen Lerner,et al.  Recent progress in road and lane detection: a survey , 2012, Machine Vision and Applications.

[7]  James M. Coughlan,et al.  Detecting and locating crosswalks using a camera phone , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[8]  Yoshua Bengio,et al.  ReSeg: A Recurrent Neural Network-Based Model for Semantic Segmentation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[9]  Jan Dirk Wegner,et al.  A Higher-Order CRF Model for Road Network Extraction , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[11]  Ramesh Raskar,et al.  Robocodes: Towards Generative Street Addresses from Satellite Imagery , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[12]  Ruofei Zhong,et al.  Automatic extraction of pavement markings on streets from point cloud data of mobile LiDAR , 2017 .

[13]  Shi,et al.  A Fast Algorithm for Finding Crosswalks using Figure-Ground Segmentation , 2006 .

[14]  Tee-Ann Teo,et al.  Reconstruction of Complex Buildings using LIDAR and 2D Maps , 2006, 3D-GIS.

[15]  Lixin Fan,et al.  Comprehensive Automated 3D Urban Environment Modelling Using Terrestrial Laser Scanning Point Cloud , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[16]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Anton Kummert,et al.  On visual crosswalk detection for driver assistance systems , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[18]  Yanming Feng,et al.  Towards an automatic system for road lane marking extraction in large-scale aerial images acquired over rural areas by hierarchical image analysis and Gabor filter , 2012 .

[19]  Roberto Cipolla,et al.  Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding , 2015, BMVC.

[20]  Roberto Manduchi,et al.  Zebra Crossing Spotter: Automatic Population of Spatial Databases for Increased Safety of Blind Travelers , 2015, ASSETS.

[21]  Dragan Ahmetovic,et al.  ZebraRecognizer: Pedestrian crossing recognition for people with visual impairment or blindness , 2016, Pattern Recognit..

[22]  Vidya N. Murali,et al.  DeepLanes: End-To-End Lane Position Estimation Using Deep Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[23]  Rodrigo F. Berriel,et al.  Deep Learning-Based Large-Scale Automatic Satellite Crosswalk Classification , 2017, IEEE Geoscience and Remote Sensing Letters.

[24]  Jiman Kim,et al.  End-To-End Ego Lane Estimation Based on Sequential Transfer Learning for Self-Driving Cars , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  S. Kurath,et al.  OSMDeepOD - Object Detection on Orthophotos with and for VGI , 2017 .

[26]  Min Bai,et al.  TorontoCity: Seeing the World with a Million Eyes , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[27]  Sanja Fidler,et al.  HD Maps: Fine-Grained Road Segmentation by Parsing Ground and Aerial Images , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  M. Jarzabek-Rychard,et al.  RECONSTRUCTION OF BUILDING OUTLINES IN DENSE URBAN AREAS BASED ON LIDAR DATA AND ADDRESS POINTS , 2012 .

[29]  Eugenio Culurciello,et al.  LinkNet: Exploiting encoder representations for efficient semantic segmentation , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[30]  Marc Pollefeys,et al.  Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark , 2017, ArXiv.

[31]  John W. Fisher,et al.  Automatic registration of LIDAR and optical images of urban scenes , 2009, CVPR.

[32]  Gerd Wanielik,et al.  Multi-channel lidar processing for lane detection and estimation , 2009, 2009 12th International IEEE Conference on Intelligent Transportation Systems.

[33]  Raquel Urtasun,et al.  DeepRoadMapper: Extracting Road Topology from Aerial Images , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[34]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Robust Semantic Pixel-Wise Labelling , 2015, CVPR 2015.

[35]  S. Kammel,et al.  Lidar-based lane marker detection and mapping , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[36]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Rainer Stiefelhagen,et al.  Zebra Crossing Detection from Aerial Imagery Across Countries , 2016, ICCHP.

[38]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Lixin Fan,et al.  Urban 3D segmentation and modelling from street view images and LiDAR point clouds , 2017, Machine Vision and Applications.

[40]  Dragan Ahmetovic,et al.  Zebralocalizer: identification and localization of pedestrian crossings , 2011, Mobile HCI.

[41]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[42]  Sanja Fidler,et al.  Enhancing Road Maps by Parsing Aerial Images Around the World , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).