Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation

Driver-assistance systems that monitor driver intent, warn drivers of lane departures, or assist in vehicle guidance are all being actively considered. It is therefore important to take a critical look at key aspects of these systems, one of which is lane-position tracking. It is for these driver-assistance objectives that motivate the development of the novel "video-based lane estimation and tracking" (VioLET) system. The system is designed using steerable filters for robust and accurate lane-marking detection. Steerable filters provide an efficient method for detecting circular-reflector markings, solid-line markings, and segmented-line markings under varying lighting and road conditions. They help in providing robustness to complex shadowing, lighting changes from overpasses and tunnels, and road-surface variations. They are efficient for lane-marking extraction because by computing only three separable convolutions, we can extract a wide variety of lane markings. Curvature detection is made more robust by incorporating both visual cues (lane markings and lane texture) and vehicle-state information. The experiment design and evaluation of the VioLET system is shown using multiple quantitative metrics over a wide variety of test conditions on a large test path using a unique instrumented vehicle. A justification for the choice of metrics based on a previous study with human-factors applications as well as extensive ground-truth testing from different times of day, road conditions, weather, and driving scenarios is also presented. In order to design the VioLET system, an up-to-date and comprehensive analysis of the current state of the art in lane-detection research was first performed. In doing so, a comparison of a wide variety of methods, pointing out the similarities and differences between methods as well as when and where various methods are most useful, is presented

[1]  Paul Milgram,et al.  The Development of a Time-Related Measure to Describe Driving Strategy , 1984 .

[2]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Ernst D. Dickmanns,et al.  Recursive 3-D Road and Relative Ego-State Recognition , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Charles E. Thorpe,et al.  The YARF system for vision-based road following , 1995 .

[5]  Sridhar Lakshmanan,et al.  A deformable-template approach to lane detection , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[6]  Shumeet Baluja,et al.  Evolution of an artificial neural network based autonomous land vehicle controller , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[7]  Todd Jochem,et al.  Rapidly Adapting Machine Vision for Automated Vehicle Steering , 1996, IEEE Expert.

[8]  Karl C. Kluge.  Performance evaluation of vision-based lane sensing: some preliminary tools, metrics, and results , 1997, Proceedings of Conference on Intelligent Transportation Systems.

[9]  Dean A. Pomerleau,et al.  Neural Network Vision for Robot Driving , 1997 .

[10]  C. Kreucher,et al.  A DRIVER WARNING SYSTEM BASED ON THE LOIS LANE DETECTION ALGORITHM , 1998 .

[11]  Massimo Bertozzi,et al.  GOLD: a parallel real-time stereo vision system for generic obstacle and lane detection , 1998, IEEE Trans. Image Process..

[12]  Jitendra Malik,et al.  A Comparative Study of Vision-Based Lateral Control Strategies for Autonomous Highway Driving , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[13]  Christopher M. Kreucher,et al.  LANA: a lane extraction algorithm that uses frequency domain features , 1999, IEEE Trans. Robotics Autom..

[14]  Alfred O. Hero,et al.  Simultaneous detection of lane and pavement boundaries using model-based multisensor fusion , 2000, IEEE Trans. Intell. Transp. Syst..

[15]  Tomohiro Yamamura,et al.  A Driver Behavior Recognition Method Based on a Driver Model Framework , 2000 .

[16]  Wilfried Enkelmann,et al.  A video-based lane keeping assistant , 2000, Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No.00TH8511).

[17]  Bernd Hürtgen,et al.  Lane following combining vision and DGPS , 2000, Image Vis. Comput..

[18]  Romuald Aufrère,et al.  Recovering the 3D shape of a road by on-board monocular vision , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[19]  Camillo J. Taylor,et al.  Stochastic Road Shape Estimation , 2001, ICCV.

[20]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[21]  Sukhan Lee,et al.  Robust lane keeping from novel sensor fusion , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[22]  Yoshiki Kobayashi,et al.  Multitype lane markers recognition using local edge direction , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[23]  Joon Woong Lee,et al.  A Machine Vision System for Lane-Departure Detection , 2002, Comput. Vis. Image Underst..

[24]  Sukhan Lee,et al.  Performance evaluation of decision making strategies for an embedded lane departure warning system , 2002, J. Field Robotics.

[25]  Massimo Bertozzi,et al.  Artificial vision in road vehicles , 2002, Proc. IEEE.

[26]  Gerd Wanielik,et al.  A new approach for lane departure identification , 2003, IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No.03TH8683).

[27]  Alexander Zelinsky,et al.  Robust vision based lane tracking using multiple cues and particle filtering , 2003, IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No.03TH8683).

[28]  Michalis E. Zervakis,et al.  A survey of video processing techniques for traffic applications , 2003, Image Vis. Comput..

[29]  Tarak Gandhi,et al.  Driver's view and vehicle surround estimation using omnidirectional video stream , 2003, IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No.03TH8683).

[30]  Dong Joong Kang,et al.  Road lane segmentation using dynamic programming for active safety vehicles , 2003, Pattern Recognit. Lett..

[31]  Mohan M. Trivedi,et al.  Occupant posture analysis with stereo and thermal infrared video: algorithms and experimental evaluation , 2004, IEEE Transactions on Vehicular Technology.

[32]  Dario D. Salvucci.  Inferring Driver Intent: A Case Study in Lane-Change Detection , 2004 .

[33]  Dinggang Shen,et al.  Lane detection and tracking using B-Snake , 2004, Image Vis. Comput..

[34]  S. Nedevschi,et al.  3D lane detection system based on stereovision , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[35]  E. Boer,et al.  A collaborative approach for human-centered driver assistance systems , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[36]  Wilfried Enkelmann,et al.  Video-Based Driver Assistance--From Basic Functions to Applications , 2001, International Journal of Computer Vision.

[37]  Tarak Gandhi,et al.  Parametric ego-motion estimation for vehicle surround analysis using an omnidirectional camera , 2005, Machine Vision and Applications.

[38]  Hans-Hellmut Nagel,et al.  Towards Active Machine-Vision-Based Driver Assistance for Urban Areas , 2002, International Journal of Computer Vision.

[39]  M.M. Trivedi,et al.  Design of an instrumented vehicle test bed for developing a human centered driver support system , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[40]  M.M. Trivedi,et al.  Visual context capture and analysis for driver attention monitoring , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[41]  Nanning Zheng,et al.  Springrobot: a prototype autonomous vehicle and its algorithms for lane detection , 2004, IEEE Transactions on Intelligent Transportation Systems.

[42]  Mohan M. Trivedi,et al.  Lane Change Intent Analysis Using Robust Operators and Sparse Bayesian Learning , 2005, IEEE Transactions on Intelligent Transportation Systems.