Road detection based on simultaneous deep learning approaches

Abstract One of the most important challenges for Autonomous Driving and Driving Assistance systems is the detection of the road to perform or monitor navigation. Many works can be found in the literature to perform road and lane detection, using both algorithmic processing and learning based techniques. However, no single solution is mentioned to be applicable in any circumstance of mixed scenarios of structured, unstructured, lane based, line based or curb based limits, and other sorts of boundaries. So, one way to embrace this challenge is to have multiple techniques, each specialized on a different approach, and combine them to obtain the best solution from individual contributions. That is the central concern of this paper. By improving a previously developed architecture to combine multiple data sources, a solution is proposed to merge the outputs of two Deep Learning based techniques for road detection. A new representation for the road is proposed along with a workflow of procedures for the combination of two simultaneous Deep Learning models, based on two adaptations of the ENet model. The results show that the overall solution copes with the alternate failures or under-performances of each model, producing a road detection result that is more reliable than the one given by each approach individually.

[1]  Fei Luo,et al.  RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic Segmentation , 2018, ArXiv.

[2]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Eugenio Culurciello,et al.  ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation , 2016, ArXiv.

[4]  Christopher Zach,et al.  ContextNet: Exploring Context and Detail for Semantic Segmentation in Real-time , 2018, BMVC.

[5]  Yu Wang,et al.  ESNet: An Efficient Symmetric Network for Real-time Semantic Segmentation , 2019, PRCV.

[6]  Denis Gingras,et al.  Autonomous Vehicle and Real Time Road Lanes Detection and Tracking , 2015, 2015 IEEE Vehicle Power and Propulsion Conference (VPPC).

[7]  Luc Van Gool,et al.  Towards End-to-End Lane Detection: an Instance Segmentation Approach , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[8]  Roberto Cipolla,et al.  Fast-SCNN: Fast Semantic Segmentation Network , 2019, BMVC.

[9]  Xiaojuan Qi,et al.  ICNet for Real-Time Semantic Segmentation on High-Resolution Images , 2017, ECCV.

[10]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Jianping Zeng,et al.  Effective Road Lane Detection and Tracking Method Using Line Segment Detector , 2018, 2018 37th Chinese Control Conference (CCC).

[12]  S. Khan,et al.  Real time lane detection for autonomous vehicles , 2008, 2008 International Conference on Computer and Communication Engineering.

[13]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Zhuoping Yu,et al.  Detecting Lane and Road Markings at A Distance with Perspective Transformer Layers , 2020, 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC).

[15]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Vitor Santos,et al.  Scalable ROS-Based Architecture to Merge Multi-source Lane Detection Algorithms , 2019, ROBOT.

[17]  Xinming Huang,et al.  Road Segmentation Using CNN with GRU , 2018, ArXiv.

[18]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[19]  Sébastien Ourselin,et al.  Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations , 2017, DLMIA/ML-CDS@MICCAI.

[20]  Luc Van Gool,et al.  Semantic Instance Segmentation with a Discriminative Loss Function , 2017, ArXiv.

[21]  Hsueh-Ming Hang,et al.  DSNet: An Efficient CNN for Road Scene Segmentation , 2019, 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC).

[22]  Sridhar Lakshmanan,et al.  A deformable-template approach to lane detection , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[23]  Samira Pouyanfar,et al.  Semantic Event Detection Using Ensemble Deep Learning , 2016, 2016 IEEE International Symposium on Multimedia (ISM).

[24]  Aybars Ugur,et al.  Image classification using ensemble algorithms with deep learning and hand-crafted features , 2018, 2018 26th Signal Processing and Communications Applications Conference (SIU).

[25]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[26]  Hsueh-Ming Hang,et al.  Multi-Class Lane Semantic Segmentation using Efficient Convolutional Networks , 2019, 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP).

[27]  Xinming Huang,et al.  RoadNet-v2: A 10 ms Road Segmentation Using Spatial Sequence Layer , 2018, ArXiv.

[28]  Changzheng Hou,et al.  An efficient lane markings detection and tracking method based on vanishing point constraints , 2016, CCC 2016.

[29]  Chen Change Loy,et al.  Learning Lightweight Lane Detection CNNs by Self Attention Distillation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[30]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Mohamed Aly,et al.  Real time detection of lane markers in urban streets , 2008, 2008 IEEE Intelligent Vehicles Symposium.