Real-time lane departure warning system based on a single FPGA

AbstractThis paper presents a camera-based lane departure warning system implemented on a field programmable gate array (FPGA) device. The system is used as a driver assistance system, which effectively prevents accidents given that it is endowed with the advantages of FPGA technology, including high performance for digital image processing applications, compactness, and low cost. The main contributions of this work are threefold. (1) An improved vanishing point-based steerable filter is introduced and implemented on an FPGA device. Using the vanishing point to guide the orientation at each pixel, this algorithm works well in complex environments. (2) An improved vanishing point-based parallel Hough transform is proposed. Unlike the traditional Hough transform, our improved version moves the coordinate origin to the estimated vanishing point to reduce storage requirements and enhance detection capability. (3) A prototype based on the FPGA is developed. With improvements in the vanishing point-based steerable filter and vanishing point-based parallel Hough transform, the prototype can be used in complex weather and lighting conditions. Experiments conducted on an evaluation platform and on actual roads illustrate the effective performance of the proposed system.

[1]  Larry Smith,et al.  Lane Departure Warning System Research and Test Development , 2007 .

[2]  G. Wanielik,et al.  Hough transformation based approach for road border detection in infrared images , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[3]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Hong Wang,et al.  Real-time lane detection in various conditions and night cases , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[5]  Lian Xiaomin.  Lane Detection Method by Using Steerable Filters , 2008 .

[6]  Nanning Zheng,et al.  Springrobot: a prototype autonomous vehicle and its algorithms for lane detection , 2004, IEEE Transactions on Intelligent Transportation Systems.

[7]  R. Anvari.  FPGA Implementation of the Lane Detection and Tracking Algorithm , 2010 .

[8]  John McDonald,et al.  Application of the Hough Transform to Lane Detection and Following on High Speed Roads , 2001 .

[9]  Amnon Shashua,et al.  A Computer Vision System on a Chip: a case study from the automotive domain , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[10]  Qiang Chen,et al.  A Real-time Lane Detection Algorithm Based on a Hyperbola-Pair Model , 2006, 2006 IEEE Intelligent Vehicles Symposium.

[11]  Jay A. Farrell,et al.  Optimization-based road curve fitting , 2011, IEEE Conference on Decision and Control and European Control Conference.

[12]  Li-Chen Fu,et al.  A Portable Vision-Based Real-Time Lane Departure Warning System: Day and Night , 2009, IEEE Transactions on Vehicular Technology.

[13]  Manuel Menezes de Oliveira Neto,et al.  Real-time line detection through an improved Hough transform voting scheme , 2008, Pattern Recognit..

[14]  Pierre Charbonnier,et al.  Evaluation of Road Marking Feature Extraction , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[15]  Kwanghoon Sohn,et al.  Illumination invariant road detection based on learning method , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[16]  Jae Wook Jeon,et al.  A real-time finite line detection system based on FPGA , 2008, 2008 6th IEEE International Conference on Industrial Informatics.

[17]  Vittorio Murino,et al.  A real-time versatile roadway path extraction and tracking on an FPGA platform , 2010, Comput. Vis. Image Underst..

[18]  M.M. Trivedi,et al.  An integrated, robust approach to lane marking detection and lane tracking , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[19]  Shengdong Pan,et al.  Content-based Auto Exposure Control for On-board CMOS Camera , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[20]  Yingfeng Cai,et al.  A Lane Departure Warning System Based on Machine Vision , 2008, 2008 IEEE Pacific-Asia Workshop on Computational Intelligence and Industrial Application.

[21]  Pierre Charbonnier,et al.  Detection and recognition of urban road markings using images , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[22]  Nam Ik Cho,et al.  Fast lane detection & tracking based on Hough transform with reduced memory requirement , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[23]  Mohan M. Trivedi,et al.  Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation , 2006, IEEE Transactions on Intelligent Transportation Systems.

[24]  Filippo Sorbello,et al.  An Embedded Real-Time Lane-Keeper for Automatic Vehicle Driving , 2008, 2008 International Conference on Complex, Intelligent and Software Intensive Systems.

[25]  Junqiang Xi,et al.  A novel lane detection based on geometrical model and Gabor filter , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[26]  Elli Angelopoulou,et al.  On feature templates for Particle Filter based lane detection , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[27]  Ming-Yang Chern,et al.  Design and Integration of Parallel Hough-Transform Chips for High-speed Line Detection , 2005, 11th International Conference on Parallel and Distributed Systems (ICPADS'05).

[28]  Rachid Deriche,et al.  Using Canny's criteria to derive a recursively implemented optimal edge detector , 1987, International Journal of Computer Vision.

[29]  Wen-Chung Kao,et al.  An embedded lane departure warning system , 2011, 2011 IEEE 15th International Symposium on Consumer Electronics (ISCE).

[30]  Richard Egli,et al.  Old and new straight-line detectors: Description and comparison , 2008, Pattern Recognit..

[31]  Joan Serrat,et al.  Robust lane markings detection and road geometry computation , 2010 .

[32]  Alexander Zelinsky,et al.  Robust vision based lane tracking using multiple cues and particle filtering , 2003, IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No.03TH8683).