Enhanced Detection and Recognition of Road Markings Based on Adaptive Region of Interest and Deep Learning

The accurate detection and classification of road markings is required for autonomous vehicles. There have been several previous studies on the detection of road lane markings, but the detection and classification of arrows and bike markings has not received much attention. There exists previous research on the detection and classification of arrows and bike markings, but they comprise a performance limitation owing to the use of the entire input image. Therefore, our approach is focused on enhancing the performance of the detection and classification of arrows and bike markings based on the adaptive region of interest (ROI) and deep convolutional neural network (CNN). In the first stage, a vanishing point is detected in order to create the ROI image. The ROI image that covers the majority of the road region is then used as the input to train the CNN-based detector and classifier in the second stage. The proposed approach is evaluated using three open datasets, namely, the Cambridge dataset, Daimler dataset, and Malaga urban dataset on a desktop computer and NVIDIA Jetson TX2 embedded system. The experimental results show that the proposed method outperforms the state-of-the-art methods in recognition performance even with small road markings at a large distance.

[1]  Xinming Huang,et al.  Road marking detection and classification using machine learning algorithms , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[2]  Weihong Deng,et al.  Very deep convolutional neural network based image classification using small training sample size , 2015, 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR).

[3]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[4]  Wen Wang,et al.  A Novel Line Space Voting Method for Vanishing-Point Detection of General Road Images , 2016, Sensors.

[5]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Ioana Maria Chira,et al.  Real-time detection of road markings for driving assistance applications , 2010, The 2010 International Conference on Computer Engineering & Systems.

[7]  In So Kweon,et al.  VPGNet: Vanishing Point Guided Network for Lane and Road Marking Detection and Recognition , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[9]  Zhao Li,et al.  Road markings extraction based on threshold segmentation , 2012, 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery.

[10]  Hayder Radha,et al.  Deep learning algorithm for autonomous driving using GoogLeNet , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[11]  Dacheng Tao,et al.  Deep Neural Network for Structural Prediction and Lane Detection in Traffic Scene , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[12]  In-So Kweon,et al.  AttentionNet: Aggregating Weak Directions for Accurate Object Detection , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[13]  Kwanghoon Sohn,et al.  Gradient-Enhancing Conversion for Illumination-Robust Lane Detection , 2013, IEEE Transactions on Intelligent Transportation Systems.

[14]  Wei Liu,et al.  DSSD : Deconvolutional Single Shot Detector , 2017, ArXiv.

[15]  Kang Ryoung Park,et al.  Road Lane Detection Robust to Shadows Based on a Fuzzy System Using a Visible Light Camera Sensor , 2017, Sensors.

[16]  Luis Herranz,et al.  Heterogeneous Convolutional Neural Networks for Visual Recognition , 2016, PCM.

[17]  Hua Li,et al.  Inverse perspective mapping based Urban road markings detection , 2012, 2012 IEEE 2nd International Conference on Cloud Computing and Intelligence Systems.

[18]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Cuneyt Akinlar,et al.  Video-Based Lane Detection Using a Fast Vanishing Point Estimation Method , 2012, 2012 IEEE International Symposium on Multimedia.

[20]  Kang Ryoung Park,et al.  Road Lane Detection by Discriminating Dashed and Solid Road Lanes Using a Visible Light Camera Sensor , 2016, Sensors.

[21]  Marcelo H. Ang,et al.  Perception, Planning, Control, and Coordination for Autonomous Vehicles , 2017 .

[22]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[24]  Anis Koubaa,et al.  Car Detection using Unmanned Aerial Vehicles: Comparison between Faster R-CNN and YOLOv3 , 2018, 2019 1st International Conference on Unmanned Vehicle Systems-Oman (UVS).

[25]  Pierre Charbonnier,et al.  Detection and recognition of urban road markings using images , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[26]  Radu Tudor Ionescu,et al.  Optimizing the Trade-Off between Single-Stage and Two-Stage Deep Object Detectors using Image Difficulty Prediction , 2018, 2018 20th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC).

[27]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[28]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Joseph Redmon,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[30]  A. Hussain,et al.  Implementation of inverse perspective mapping algorithm for the development of an automatic lane tracking system , 2004, 2004 IEEE Region 10 Conference TENCON 2004..

[31]  Dumitru Erhan,et al.  Scalable Object Detection Using Deep Neural Networks , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Rafael Grompone von Gioi,et al.  LSD: a Line Segment Detector , 2012, Image Process. Line.

[33]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[35]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[36]  Yan Wang,et al.  Pseudo-LiDAR From Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  In So Kweon,et al.  Robust Road Marking Detection and Recognition Using Density-Based Grouping and Machine Learning Techniques , 2017, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[39]  S. Colman,et al.  Generated Traffic : Implications for Transport Planning , 2001 .

[40]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Larry S. Davis,et al.  G-CNN: An Iterative Grid Based Object Detector , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[43]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[44]  Kang Ryoung Park,et al.  Deep RetinaNet-Based Detection and Classification of Road Markings by Visible Light Camera Sensors , 2019, Sensors.

[45]  Yang Yan,et al.  Accurate and robust lane detection based on Dual-View Convolutional Neutral Network , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[46]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Miguel Oliveira,et al.  Multimodal inverse perspective mapping , 2015, Inf. Fusion.

[48]  Kil To Chong,et al.  Object Detection in Very High-Resolution Aerial Images Using One-Stage Densely Connected Feature Pyramid Network , 2018, Sensors.

[49]  Shang-Jeng Tsai,et al.  HSI color model based lane-marking detection , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[50]  Zhiqiang Shen,et al.  DSOD: Learning Deeply Supervised Object Detectors from Scratch , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).