Comprehensive and Practical Vision System for Self-Driving Vehicle Lane-Level Localization

Vehicle lane-level localization is a fundamental technology in autonomous driving. To achieve accurate and consistent performance, a common approach is to use the LIDAR technology. However, it is expensive and computational demanding, and thus not a practical solution in many situations. This paper proposes a stereovision system, which is of low cost, yet also able to achieve high accuracy and consistency. It integrates a new lane line detection algorithm with other lane marking detectors to effectively identify the correct lane line markings. It also fits multiple road models to improve accuracy. An effective stereo 3D reconstruction method is proposed to estimate vehicle localization. The estimation consistency is further guaranteed by a new particle filter framework, which takes vehicle dynamics into account. Experiment results based on image sequences taken under different visual conditions showed that the proposed system can identify the lane line markings with 98.6% accuracy. The maximum estimation error of the vehicle distance to lane lines is 16 cm in daytime and 26 cm at night, and the maximum estimation error of its moving direction with respect to the road tangent is 0.06 rad in daytime and 0.12 rad at night. Due to its high accuracy and consistency, the proposed system can be implemented in autonomous driving vehicles as a practical solution to vehicle lane-level localization.

[1]  Ronald N. Bracewell,et al.  The Fourier Transform and Its Applications , 1966 .

[2]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[3]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[4]  Romuald Aufrère,et al.  A model-driven approach for real-time road recognition , 2001, Machine Vision and Applications.

[5]  Dinggang Shen,et al.  Lane detection and tracking using B-Snake , 2004, Image Vis. Comput..

[6]  R. Sukthankar,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[7]  Jerome Douret,et al.  A Reliable and Robust Lane Detection System based on the Parallel Use of Three Algorithms for Driving Safety Assistance , 2006, MVA.

[8]  Jin Wang,et al.  Lane keeping based on location technology , 2005, IEEE Transactions on Intelligent Transportation Systems.

[9]  Shang-Jeng Tsai,et al.  HSI color model based lane-marking detection , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[10]  Minoru Okada,et al.  A Road Extraction Method by an Active Contour Model with Inertia and Differential Features , 2006, IEICE Trans. Inf. Syst..

[11]  Jerome Douret,et al.  A multi-model lane detector that handles road singularities , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[12]  Mohan M. Trivedi,et al.  Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation , 2006, IEEE Transactions on Intelligent Transportation Systems.

[13]  Bing-Fei Wu,et al.  The Heterogeneous Systems Integration Design and Implementation for Lane Keeping on a Vehicle , 2008, IEEE Transactions on Intelligent Transportation Systems.

[14]  Giuseppe Messina,et al.  Exposure Correction for Imaging Devices: An Overview , 2008 .

[15]  ZuWhan Kim,et al.  Robust Lane Detection and Tracking in Challenging Scenarios , 2008, IEEE Transactions on Intelligent Transportation Systems.

[16]  Monson H. Hayes,et al.  Robust lane detection and tracking with ransac and Kalman filter , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[17]  Sergiu Nedevschi,et al.  Probabilistic Lane Tracking in Difficult Road Scenarios Using Stereovision , 2009, IEEE Transactions on Intelligent Transportation Systems.

[18]  Seiichi Mita,et al.  Stereovision-based road boundary detection for intelligent vehicles in challenging scenarios , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Le Thanh Sach,et al.  A robust road profile estimation method for low texture stereo images , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[20]  Shigang Wang,et al.  New Lane Model and Distance Transform for Lane Detection and Tracking , 2009, CAIP.

[21]  Sebastian Thrun,et al.  Junior: The Stanford entry in the Urban Challenge , 2008, J. Field Robotics.

[22]  Luo Juan,et al.  A comparison of SIFT, PCA-SIFT and SURF , 2009 .

[23]  Edwin Olson,et al.  Finding multiple lanes in urban road networks with vision and lidar , 2009, Auton. Robots.

[24]  Joan Serrat,et al.  Robust lane markings detection and road geometry computation , 2010 .

[25]  Shigang Wang,et al.  Lane detection and tracking using a new lane model and distance transform , 2011, Mach. Vis. Appl..

[26]  Guoyan Xu,et al.  Computer vision-based multiple-lane detection on straight road and in a curve , 2010, 2010 International Conference on Image Analysis and Signal Processing.

[27]  Tom Drummond,et al.  Faster and Better: A Machine Learning Approach to Corner Detection , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Guido Perboli,et al.  Investigating the impact of a Lane Departure Warning system in real driving conditions - A subjective Field Operational Test , 2010 .

[29]  Ronen Lerner,et al.  Recent progress in road and lane detection: a survey , 2012, Machine Vision and Applications.

[30]  Yo-Sung Ho,et al.  An efficient image rectification method for parallel multi-camera arrangement , 2011, IEEE Transactions on Consumer Electronics.

[31]  Mark E. Campbell,et al.  Map‐aided localization in sparse global positioning system environments using vision and particle filtering , 2011, J. Field Robotics.

[32]  Fawzi Nashashibi,et al.  Robust real-time lane detection based on lane mark segment features and general a priori knowledge , 2011, 2011 IEEE International Conference on Robotics and Biomimetics.

[33]  Monson H. Hayes,et al.  A Novel Lane Detection System With Efficient Ground Truth Generation , 2012, IEEE Transactions on Intelligent Transportation Systems.

[34]  Tao Wu,et al.  A practical system for road marking detection and recognition , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[35]  A. Goshtasby.  Similarity and Dissimilarity Measures , 2012 .

[36]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  S. Govindarajulu,et al.  A Comparison of SIFT, PCA-SIFT and SURF , 2012 .

[38]  Mohan M. Trivedi,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 1 Integrated Lane and Vehicle Detection, Localization, , 2022 .

[39]  Whoi-Yul Kim,et al.  Lane detection system with around view monitoring for intelligent vehicle , 2013, 2013 International SoC Design Conference (ISOCC).

[40]  Florentin Wörgötter,et al.  Stochastic Lane Shape Estimation Using Local Image Descriptors , 2013, IEEE Transactions on Intelligent Transportation Systems.

[41]  Jannik Fritsch,et al.  A new performance measure and evaluation benchmark for road detection algorithms , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[42]  Kwanghoon Sohn,et al.  Gradient-Enhancing Conversion for Illumination-Robust Lane Detection , 2013, IEEE Transactions on Intelligent Transportation Systems.

[43]  Jing Li,et al.  Robust multilane detection and tracking in urban scenarios based on LIDAR and mono-vision , 2014, IET Image Process..

[44]  Qingquan Li,et al.  A Sensor-Fusion Drivable-Region and Lane-Detection System for Autonomous Vehicle Navigation in Challenging Road Scenarios , 2014, IEEE Transactions on Vehicular Technology.

[45]  Christopher Rose,et al.  An Integrated Vehicle Navigation System Utilizing Lane-Detection and Lateral Position Estimation Systems in Difficult Environments for GPS , 2014, IEEE Transactions on Intelligent Transportation Systems.

[46]  Emmanuel Seignez,et al.  Lane marking based vehicle localization using particle filter and multi-kernel estimation , 2014, 2014 13th International Conference on Control Automation Robotics & Vision (ICARCV).

[47]  Soon Kwon,et al.  Various lane marking detection and classification for vision-based navigation system? , 2015, 2015 IEEE International Conference on Consumer Electronics (ICCE).

[48]  Kok Kiong Tan,et al.  Vision-based lane line detection for autonomous vehicle navigation and guidance , 2015, 2015 10th Asian Control Conference (ASCC).

[49]  Kok Kiong Tan,et al.  Vision-based approach towards lane line detection and vehicle localization , 2015, Machine Vision and Applications.

[50]  Kok Kiong Tan,et al.  Autonomous Reverse Parking System Based on Robust Path Generation and Improved Sliding Mode Control , 2015, IEEE Transactions on Intelligent Transportation Systems.

[51]  Christoph Stiller,et al.  Efficient Road Scene Understanding for Intelligent Vehicles Using Compositional Hierarchical Models , 2015, IEEE Transactions on Intelligent Transportation Systems.

[52]  Diana Adler,et al.  Single Sensor Imaging Methods And Applications For Digital Cameras , 2016 .