Lane Detection and Classification for Forward Collision Warning System Based on Stereo Vision

This paper presents a lightweight stereo vision-based driving lane detection and classification system to achieve the ego-car’s lateral positioning and forward collision warning to aid advanced driver assistance systems (ADAS). For lane detection, we design a self-adaptive traffic lanes model in Hough Space with a maximum likelihood angle and dynamic pole detection region of interests (ROIs), which is robust to road bumpiness, lane structure changing while the ego-car’s driving and interferential markings on the ground. What’s more, this model can be improved with geographic information system or electronic map to achieve more accurate results. Besides, the 3-D information acquired by stereo matching is used to generate an obstacle mask to reduce irrelevant objects’ interfere and detect forward collision distance. For lane classification, a convolutional neural network is trained by using manually labeled ROI from KITTI data set to classify the left/right-side line of host lane so that we can provide significant information for lane changing strategy making in ADAS. Quantitative experimental evaluation shows good true positive rate on lane detection and classification with a real-time (15Hz) working speed. Experimental results also demonstrate a certain level of system robustness on variation of the environment.

[1]  Dinggang Shen,et al.  Lane detection and tracking using B-Snake , 2004, Image Vis. Comput..

[2]  Sibel Yenikaya,et al.  Keeping the vehicle on the road: A survey on on-road lane detection systems , 2013, CSUR.

[3]  Gerd Wanielik,et al.  Situation Assessment for Automatic Lane-Change Maneuvers , 2010, IEEE Transactions on Intelligent Transportation Systems.

[4]  Jussi Parkkinen,et al.  Real-Time Lane Detection and Rear-End Collision Warning System on a Mobile Computing Platform , 2015, 2015 IEEE 39th Annual Computer Software and Applications Conference.

[5]  Mengyin Fu,et al.  Multi-lanes detection based on panoramic camera , 2014, 11th IEEE International Conference on Control & Automation (ICCA).

[6]  Michael Anthony Bauer,et al.  Map-based lane and obstacle-free area detection , 2015, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[7]  Fernando A. Mujica,et al.  An Empirical Evaluation of Deep Learning on Highway Driving , 2015, ArXiv.

[8]  Philomina Simon,et al.  Advanced Driver Assistance System , 2020, International Journal of Recent Technology and Engineering.

[9]  Monson H. Hayes,et al.  A Novel Lane Detection System With Efficient Ground Truth Generation , 2012, IEEE Transactions on Intelligent Transportation Systems.

[10]  Mohamed Aly,et al.  Real time detection of lane markers in urban streets , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[11]  Apurba Das,et al.  Enhanced Algorithm of Automated Ground Truth Generation and Validation for Lane Detection System by $\text{M}^{2}\text{BMT}$ , 2017, IEEE Transactions on Intelligent Transportation Systems.

[12]  Yi Yang,et al.  Real-Time Obstacles Detection and Status Classification for Collision Warning in a Vehicle Active Safety System , 2018, IEEE Transactions on Intelligent Transportation Systems.

[13]  Dinesh Babu Jayagopi,et al.  A robust lane detection and departure warning system , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[14]  Alexander Verl,et al.  Vision-based robust road lane detection in urban environments , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[15]  Sanghoon Sull,et al.  Efficient Lane Detection Based on Spatiotemporal Images , 2016, IEEE Transactions on Intelligent Transportation Systems.

[16]  Ronen Lerner,et al.  Recent progress in road and lane detection: a survey , 2012, Machine Vision and Applications.

[17]  Huanling Wang,et al.  Local Stereo Matching Based on Support Weight With Motion Flow for Dynamic Scene , 2016, IEEE Access.

[18]  Yang Yan,et al.  Accurate and robust lane detection based on Dual-View Convolutional Neutral Network , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[19]  Jianwei Niu,et al.  Robust Lane Detection using Two-stage Feature Extraction with Curve Fitting , 2016, Pattern Recognit..

[20]  Elli Angelopoulou,et al.  On feature templates for Particle Filter based lane detection , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[21]  Yun Liu,et al.  Research on medical applications of contrast sensitivity function to red-green gratings in 3D space , 2017, Neurocomputing.

[22]  John G. Rarity,et al.  U-V-Disparity based Obstacle Detection with 3D Camera and steerable filter , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[23]  Jun Wang,et al.  An approach of lane detection based on Inverse Perspective Mapping , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[24]  Dacheng Tao,et al.  Deep Neural Network for Structural Prediction and Lane Detection in Traffic Scene , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Yun Liu,et al.  New stereo shooting evaluation metric based on stereoscopic distortion and subjective perception , 2015 .

[26]  Jianfeng Wang,et al.  Lane detection based on random hough transform on region of interesting , 2010, The 2010 IEEE International Conference on Information and Automation.

[27]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[28]  Cláudio Rosito Jung,et al.  Automatic Detection and Classification of Road Lane Markings Using Onboard Vehicular Cameras , 2015, IEEE Transactions on Intelligent Transportation Systems.

[29]  Seung-Woo Seo,et al.  Multi-lane detection based on accurate geometric lane estimation in highway scenarios , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[30]  Andreas Geiger,et al.  Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art , 2017, Found. Trends Comput. Graph. Vis..

[31]  Rama Chellappa,et al.  A Learning Approach Towards Detection and Tracking of Lane Markings , 2012, IEEE Transactions on Intelligent Transportation Systems.

[32]  Ethan Fetaya,et al.  StixelNet: A Deep Convolutional Network for Obstacle Detection and Road Segmentation , 2015, BMVC.

[33]  Qingquan Li,et al.  A Sensor-Fusion Drivable-Region and Lane-Detection System for Autonomous Vehicle Navigation in Challenging Road Scenarios , 2014, IEEE Transactions on Vehicular Technology.

[34]  Yong Zhu,et al.  A novel curve lane detection based on Improved River Flow and RANSA , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[35]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[36]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Minho Lee,et al.  Robust Lane Detection Based On Convolutional Neural Network and Random Sample Consensus , 2014, ICONIP.

[38]  Tao Wu,et al.  Multi-lane detection based on multiple vanishing points detection , 2015, International Conference on Graphic and Image Processing.

[39]  Jiachen Yang,et al.  Stereo chromatic contrast sensitivity model to blue-yellow gratings. , 2016, Optics express.

[40]  Vidya N. Murali,et al.  DeepLanes: End-To-End Lane Position Estimation Using Deep Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[41]  Keshav Bimbraw,et al.  Autonomous cars: Past, present and future a review of the developments in the last century, the present scenario and the expected future of autonomous vehicle technology , 2015, 2015 12th International Conference on Informatics in Control, Automation and Robotics (ICINCO).

[42]  Antonio M. López,et al.  Embedded Real-time Stereo Estimation via Semi-Global Matching on the GPU , 2016, ICCS.

[43]  Naim Dahnoun,et al.  Multiple Lane Detection Algorithm Based on Novel Dense Vanishing Point Estimation , 2017, IEEE Transactions on Intelligent Transportation Systems.

[44]  Marc Pollefeys,et al.  The Stixel World: A medium-level representation of traffic scenes , 2017, Image Vis. Comput..