DeepLanes: End-To-End Lane Position Estimation Using Deep Neural Networks

Camera-based lane detection algorithms are one of the key enablers for many semi-autonomous and fullyautonomous systems, ranging from lane keep assist to level-5 automated vehicles. Positioning a vehicle between lane boundaries is the core navigational aspect of a self-driving car. Even though this should be trivial, given the clarity of lane markings on most standard roadway systems, the process is typically mired with tedious pre-processing and computational effort. We present an approach to estimate lane positions directly using a deep neural network that operates on images from laterally-mounted down-facing cameras. To create a diverse training set, we present a method to generate semi-artificial images. Besides the ability to distinguish whether there is a lane-marker present or not, the network is able to estimate the position of a lane marker with sub-centimeter accuracy at an average of 100 frames/s on an embedded automotive platform, requiring no pre-or post-processing. This system can be used not only to estimate lane position for navigation, but also provide an efficient way to validate the robustness of driver-assist features which depend on lane information.

[1]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[3]  Florentin Wörgötter,et al.  Combining Statistical Hough Transform and Particle Filter for robust lane detection and tracking , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[4]  ZuWhan Kim,et al.  Robust Lane Detection and Tracking in Challenging Scenarios , 2008, IEEE Transactions on Intelligent Transportation Systems.

[5]  Christoph Stiller,et al.  Kalman Particle Filter for lane recognition on rural roads , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[6]  Gudrun Klinker,et al.  Stable Road Lane Model Based on Clothoids , 2010 .

[7]  Vidya N. Murali,et al.  Generation and Usage of Virtual Data for the Development of Perception Algorithms Using Vision , 2016 .

[8]  Minho Lee,et al.  Robust Lane Detection Based On Convolutional Neural Network and Random Sample Consensus , 2014, ICONIP.

[9]  Zhu Teng,et al.  Real-time lane detection by using multiple cues , 2010, ICCAS 2010.

[10]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Ronen Lerner,et al.  Recent progress in road and lane detection: a survey , 2012, Machine Vision and Applications.

[12]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Rama Chellappa,et al.  A Learning Approach Towards Detection and Tracking of Lane Markings , 2012, IEEE Transactions on Intelligent Transportation Systems.

[14]  Sheng-Fuu Lin,et al.  Lane detection using color-based segmentation , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[15]  Joachim Denzler,et al.  Convolutional Patch Networks with Spatial Prior for Road Detection and Urban Scene Understanding , 2015, VISAPP.

[16]  H. Neumann,et al.  Multiple Cue Data Fusion with Particle Filters for Road Course Detection in Vision Systems , 2006, 2006 IEEE Intelligent Vehicles Symposium.

[17]  Dacheng Tao,et al.  Deep Neural Network for Structural Prediction and Lane Detection in Traffic Scene , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Junqiang Xi,et al.  A novel lane detection based on geometrical model and Gabor filter , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[19]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Mohamed Aly,et al.  Real time detection of lane markers in urban streets , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[21]  Sergiu Nedevschi,et al.  Probabilistic Lane Tracking in Difficult Road Scenarios Using Stereovision , 2009, IEEE Transactions on Intelligent Transportation Systems.

[22]  Joan Serrat,et al.  Robust lane markings detection and road geometry computation , 2010 .

[23]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Fernando A. Mujica,et al.  An Empirical Evaluation of Deep Learning on Highway Driving , 2015, ArXiv.

[25]  Sridhar Lakshmanan,et al.  A deformable-template approach to lane detection , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[26]  Xiang Chen,et al.  Road segmentation via iterative deep analysis , 2015, 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO).