Design and FPGA implementation of dual-stage lane detection, based on Hough transform and localized stripe features

Abstract Robust, accurate and real-time detection of lane boundaries based on embedded hardware is an essential element of several driver assistant systems. In this work we present an FPGA-based dual-stage lane detection algorithm to cope with real world challenges such as cast shadows, occlusion of lane markers, brightness variations, wear, etc. In first stage, Sobel operator and adaptive threshold are used to extract lane edges, followed by Hough transform to extract the road markers. Second stage of the algorithm operates on original grayscale image and identifies stripe features near several candidate points with highest probabilities to find the landmarks. These extracted features are then used to detect the lane boundaries with high accuracy. Experimental results based on FPGA platform under various road conditions obtained from various datasets indicate that our algorithm can process about 60 frames per second for 720 pixels video input. Lane detection accuracy of 94.3% is achieved in average which may reach up to 97.8% in low congested highway during daylight.

[1]  Jin Zhao,et al.  Real-time lane departure and front collision warning system on an FPGA , 2014, 2014 IEEE High Performance Extreme Computing Conference (HPEC).

[2]  Kang-Hyun Jo,et al.  Real-Time Lane Region Detection Using a Combination of Geometrical and Image Features , 2016, Sensors.

[3]  Ronen Lerner,et al.  Recent progress in road and lane detection: a survey , 2012, Machine Vision and Applications.

[4]  Yang Yan,et al.  Lane marking detection based on Convolution Neural Network from point clouds , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[5]  Kwanghoon Sohn,et al.  Gradient-Enhancing Conversion for Illumination-Robust Lane Detection , 2013, IEEE Transactions on Intelligent Transportation Systems.

[6]  Simon O'Keefe,et al.  A Multi-scale Piecewise-Linear Feature Detector for Spectrogram Tracks , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[7]  Mohamed Aly,et al.  Real time detection of lane markers in urban streets , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[8]  A. Lopez,et al.  Detection of lane markings based on ridgeness and RANSAC , 2005, Proceedings. 2005 IEEE Intelligent Transportation Systems, 2005..

[9]  Fernando Martinez Vallina.  Implementing Memory Structures for Video Processing in the Vivado HLS Tool , 2012 .

[10]  Bing-Fei Wu,et al.  The Heterogeneous Systems Integration Design and Implementation for Lane Keeping on a Vehicle , 2008, IEEE Transactions on Intelligent Transportation Systems.

[11]  Vijay Gaikwad,et al.  Lane Departure Identification for Advanced Driver Assistance , 2015, IEEE Transactions on Intelligent Transportation Systems.

[12]  Xiangjing An,et al.  A real-time lane departure warning system based on FPGA , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[13]  Xinming Huang,et al.  An FPGA co-processor for adaptive lane departure warning system , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[14]  Chung-Bin Wu,et al.  Ultra-Low Complexity Block-Based Lane Detection and Departure Warning System , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Philippe Bonnifait,et al.  Sequential Data Fusion of GNSS Pseudoranges and Dopplers With Map-Based Vision Systems , 2016, IEEE Transactions on Intelligent Vehicles.

[16]  Shutao Li,et al.  A Real-Time System for Lane Detection Based on FPGA and DSP , 2016 .

[17]  Li-Chen Fu,et al.  A Portable Vision-Based Real-Time Lane Departure Warning System: Day and Night , 2009, IEEE Transactions on Vehicular Technology.

[18]  Monson H. Hayes,et al.  A Novel Lane Detection System With Efficient Ground Truth Generation , 2012, IEEE Transactions on Intelligent Transportation Systems.