Machine Vision Based Traffic Sign Detection Methods: Review, Analyses and Perspectives

Traffic signs recognition (TSR) is an important part of some advanced driver-assistance systems (ADASs) and auto driving systems (ADSs). As the first key step of TSR, traffic sign detection (TSD) is a challenging problem because of different types, small sizes, complex driving scenes, and occlusions. In recent years, there have been a large number of TSD algorithms based on machine vision and pattern recognition. In this paper, a comprehensive review of the literature on TSD is presented. We divide the reviewed detection methods into five main categories: color-based methods, shape-based methods, color- and shape-based methods, machine-learning-based methods, and LIDAR-based methods. The methods in each category are also classified into different subcategories for understanding and summarizing the mechanisms of different methods. For some reviewed methods that lack comparisons on public datasets, we reimplemented part of these methods for comparison. The experimental comparisons and analyses are presented on the reported performance and the performance of our reimplemented methods. Furthermore, future directions and recommendations of the TSD research are given to promote the development of the TSD.

[1]  Luc Van Gool,et al.  Traffic sign recognition — How far are we from the solution? , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[2]  Said M. Easa,et al.  Potential Accuracy of Traffic Signs' Positions Extracted From Google Street View , 2013, IEEE Transactions on Intelligent Transportation Systems.

[3]  Pedro Arias,et al.  PERFORMANCE TESTING OF 3D POINT CLOUD SOFTWARE , 2013 .

[4]  Nicolai Petkov,et al.  Distance sets for shape filters and shape recognition , 2003, IEEE Trans. Image Process..

[5]  Yassine Ruichek,et al.  Classification of Traffic Signs: The European Dataset , 2018, IEEE Access.

[6]  Xiaolin Hu,et al.  Traffic sign detection based on convolutional neural networks , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[7]  Cuiping Li,et al.  Deep Saliency With Channel-Wise Hierarchical Feature Responses for Traffic Sign Detection , 2019, IEEE Transactions on Intelligent Transportation Systems.

[8]  Xiaoli Hao,et al.  Robust Traffic Sign Recognition Based on Color Global and Local Oriented Edge Magnitude Patterns , 2014, IEEE Transactions on Intelligent Transportation Systems.

[9]  Belén Riveiro,et al.  Exploiting synergies of mobile mapping sensors and deep learning for traffic sign recognition systems , 2017, Expert Syst. Appl..

[10]  Pedro Arias,et al.  Automatic Segmentation and Shape-Based Classification of Retro-Reflective Traffic Signs from Mobile LiDAR Data , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[11]  Anjan Gudigar,et al.  A review on automatic detection and recognition of traffic sign , 2014, Multimedia Tools and Applications.

[12]  Bo Ai,et al.  On the Influence of Scattering From Traffic Signs in Vehicle-to-X Communications , 2016, IEEE Transactions on Vehicular Technology.

[13]  Hilario Gómez Moreno,et al.  Traffic sign shape classification and localization based on the normalized FFT of the signature of blobs and 2D homographies , 2008 .

[14]  Sei-Wang Chen,et al.  Road-sign detection and tracking , 2003, IEEE Trans. Veh. Technol..

[15]  Hasan Fleyeh,et al.  Traffic and Road Sign Recognition , 2008 .

[16]  Cuneyt Akinlar,et al.  On circular traffic sign detection and recognition , 2016, Expert Syst. Appl..

[17]  Saturnino Maldonado-Bascón,et al.  Goal Evaluation of Segmentation Algorithms for Traffic Sign Recognition , 2010, IEEE Transactions on Intelligent Transportation Systems.

[18]  Mirko Meuter,et al.  A Decision Fusion and Reasoning Module for a Traffic Sign Recognition System , 2011, IEEE Transactions on Intelligent Transportation Systems.

[19]  Kang Kim,et al.  Simultaneous Traffic Sign Detection and Boundary Estimation Using Convolutional Neural Network , 2018, IEEE Transactions on Intelligent Transportation Systems.

[20]  Faliang Chang,et al.  Fast Traffic Sign Recognition via High-Contrast Region Extraction and Extended Sparse Representation , 2016, IEEE Transactions on Intelligent Transportation Systems.

[21]  Tao Chen,et al.  Accurate and Efficient Traffic Sign Detection Using Discriminative AdaBoost and Support Vector Regression , 2016, IEEE Transactions on Vehicular Technology.

[22]  Qi Wang,et al.  An Incremental Framework for Video-Based Traffic Sign Detection, Tracking, and Recognition , 2017, IEEE Transactions on Intelligent Transportation Systems.

[23]  Johannes Stallkamp,et al.  Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition , 2012, Neural Networks.

[24]  Xiaohui Liu,et al.  Real-time traffic sign recognition from video by class-specific discriminative features , 2010, Pattern Recognit..

[25]  Baoli Li,et al.  Traffic-Sign Detection and Classification in the Wild , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Wenyu Liu,et al.  Cascaded Segmentation-Detection Networks for Text-Based Traffic Sign Detection , 2018, IEEE Transactions on Intelligent Transportation Systems.

[27]  Thomas B. Moeslund,et al.  Vision-Based Traffic Sign Detection and Analysis for Intelligent Driver Assistance Systems: Perspectives and Survey , 2012, IEEE Transactions on Intelligent Transportation Systems.

[28]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[29]  Johannes Stallkamp,et al.  Detection of traffic signs in real-world images: The German traffic sign detection benchmark , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[30]  Andrew Zisserman,et al.  Representing shape with a spatial pyramid kernel , 2007, CIVR '07.

[31]  José Luis Rojo-Álvarez,et al.  Traffic sign segmentation and classification using statistical learning methods , 2015, Neurocomputing.

[32]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[33]  Andreas Møgelmose,et al.  Detection of U.S. Traffic Signs , 2015, IEEE Transactions on Intelligent Transportation Systems.

[34]  Faliang Chang,et al.  Rapid Multiclass Traffic Sign Detection in High-Resolution Images , 2014, IEEE Transactions on Intelligent Transportation Systems.

[35]  F. Larsson,et al.  Correlating fourier descriptors of local patches for road sign recognition , 2011 .

[36]  Xiaolin Hu,et al.  Traffic sign detection by ROI extraction and histogram features-based recognition , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[37]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Mohamed El Ansari,et al.  Traffic sign detection and recognition based on random forests , 2016, Appl. Soft Comput..

[39]  Zhilu Wu,et al.  A robust, coarse-to-fine traffic sign detection method , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[40]  Zhidong Deng,et al.  Detection and Recognition of Traffic Planar Objects Using Colorized Laser Scan and Perspective Distortion Rectification , 2018, IEEE Transactions on Intelligent Transportation Systems.

[41]  Kyung-Joong Kim,et al.  Design of a visual perception model with edge-adaptive Gabor filter and support vector machine for traffic sign detection , 2013, Expert Syst. Appl..

[42]  F. J. Acevedo-Rodríguez,et al.  TRAFFIC SIGN CLASSIFICATION INVARIANT TO ROTATIONS USING SUPPORT VECTOR MACHINES , 2007 .

[43]  Jianming Zhang,et al.  A Real-Time Chinese Traffic Sign Detection Algorithm Based on Modified YOLOv2 , 2017, Algorithms.

[44]  Cheng-Lin Liu,et al.  Traffic Sign Detection Using a Cascade Method With Fast Feature Extraction and Saliency Test , 2017, IEEE Transactions on Intelligent Transportation Systems.

[45]  Meng-Yin Fu,et al.  A survey of traffic sign recognition , 2010, 2010 International Conference on Wavelet Analysis and Pattern Recognition.

[46]  S. Lafuente-Arroyo,et al.  Color images segmentation using the Support Vector Machines , 2003 .

[47]  Fatih Murat Porikli,et al.  Fast Detection of Multiple Objects in Traffic Scenes With a Common Detection Framework , 2015, IEEE Transactions on Intelligent Transportation Systems.

[48]  Chenglu Wen,et al.  Spatial-Related Traffic Sign Inspection for Inventory Purposes Using Mobile Laser Scanning Data , 2016, IEEE Transactions on Intelligent Transportation Systems.

[49]  Ignacio Parra,et al.  Automatic Traffic Signs and Panels Inspection System Using Computer Vision , 2011, IEEE Transactions on Intelligent Transportation Systems.

[50]  Fatin Zaklouta,et al.  Real-time traffic sign recognition in three stages , 2014, Robotics Auton. Syst..

[51]  Arun Kumar Sangaiah,et al.  Deep detection network for real-life traffic sign in vehicular networks , 2018, Comput. Networks.

[52]  Yiping Chen,et al.  Traffic Sign Occlusion Detection Using Mobile Laser Scanning Point Clouds , 2017, IEEE Transactions on Intelligent Transportation Systems.

[53]  S. Xu,et al.  Robust traffic sign shape recognition using geometric matching , 2009 .

[54]  Pietro Perona,et al.  Integral Channel Features , 2009, BMVC.

[55]  Shengcai Liao,et al.  Face Detection Based on Multi-Block LBP Representation , 2007, ICB.

[56]  Federico Tombari,et al.  A traffic sign detection pipeline based on interest region extraction , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[57]  Xiaoli Hao,et al.  A Cognitively Motivated Method for Classification of Occluded Traffic Signs , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[58]  Andrés Serna,et al.  Detection, segmentation and classification of 3D urban objects using mathematical morphology and supervised learning , 2014 .

[59]  Pavel Y. Yakimov,et al.  Traffic Signs Detection Using Tracking with Prediction , 2015, ICETE.

[60]  Jiujun Cheng,et al.  Traffic sign detection based on cascaded convolutional neural networks , 2016, 2016 17th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD).

[61]  许华荣,et al.  Towards Real-Time Traffic Sign Detection and Classification , 2016 .

[62]  Cheng Wang,et al.  Bag-of-visual-phrases and hierarchical deep models for traffic sign detection and recognition in mobile laser scanning data , 2016 .

[63]  Majid Mirmehdi,et al.  Real-Time Detection and Recognition of Road Traffic Signs , 2012, IEEE Transactions on Intelligent Transportation Systems.

[64]  Jian Li,et al.  Automatic detection of road traffic signs from natural scene images based on pixel vector and central projected shape feature , 2012 .

[65]  Liang Zhong,et al.  Robust Traffic-Sign Detection and Classification Using Mobile LiDAR Data With Digital Images , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[66]  Xiaoli Hao,et al.  Traffic Sign Detection via Graph-Based Ranking and Segmentation Algorithms , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[67]  Vladimir Fursov,et al.  Traffic Signs Detection and tracking using modified Hough transform , 2015, 2015 12th International Joint Conference on e-Business and Telecommunications (ICETE).

[68]  Jesmin F. Khan,et al.  Hierarchical clustering of EMD based interest points for road sign detection , 2014 .

[69]  Pedro Arias,et al.  Traffic sign detection in MLS acquired point clouds for geometric and image-based semantic inventory , 2016 .

[70]  Nick Barnes,et al.  Real-Time Regular Polygonal Sign Detection , 2005, FSR.

[71]  Jordi Vitrià,et al.  Traffic Sign Recognition Using Evolutionary Adaboost Detection and Forest-ECOC Classification , 2009, IEEE Transactions on Intelligent Transportation Systems.

[72]  Mario Muñoz Organero,et al.  Validating the Impact on Reducing Fuel Consumption by Using an EcoDriving Assistant Based on Traffic Sign Detection and Optimal Deceleration Patterns , 2013, IEEE Transactions on Intelligent Transportation Systems.

[73]  David Shinar,et al.  The effect of context and drivers' age on highway traffic signs comprehension , 2015 .

[74]  Lijuan Liu,et al.  A novel traffic sign detection method via color segmentation and robust shape matching , 2015, Neurocomputing.

[75]  Fuchun Sun,et al.  Supervised Low-Rank Matrix Recovery for Traffic Sign Recognition in Image Sequences , 2013, IEEE Signal Processing Letters.

[76]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[77]  José Manuel Pastor,et al.  Visual sign information extraction and identification by deformable models for intelligent vehicles , 2004, IEEE Transactions on Intelligent Transportation Systems.

[78]  Valeria Vignali,et al.  Fixation distance and fixation duration to vertical road signs. , 2018, Applied ergonomics.

[79]  Michael Felsberg,et al.  Using Fourier Descriptors and Spatial Models for Traffic Sign Recognition , 2011, SCIA.

[80]  Zhao Wang,et al.  Robust chinese traffic sign detection and recognition with deep convolutional neural network , 2015, 2015 11th International Conference on Natural Computation (ICNC).

[81]  Majid Mirmehdi,et al.  Recognizing Text-Based Traffic Signs , 2015, IEEE Transactions on Intelligent Transportation Systems.

[82]  Federico Tombari,et al.  Traffic sign detection via interest region extraction , 2015, Pattern Recognit..

[83]  Gareth Blake Loy,et al.  Fast shape-based road sign detection for a driver assistance system , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[84]  Zhaohui Wu,et al.  Weakly Supervised Metric Learning for Traffic Sign Recognition in a LIDAR-Equipped Vehicle , 2016, IEEE Transactions on Intelligent Transportation Systems.

[85]  Pietro Perona,et al.  Fast Feature Pyramids for Object Detection , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[86]  Alexander Wong,et al.  MicronNet: A Highly Compact Deep Convolutional Neural Network Architecture for Real-Time Embedded Traffic Sign Classification , 2018, IEEE Access.

[87]  Christophe Cudel,et al.  Multi-ROI Association and Tracking With Belief Functions: Application to Traffic Sign Recognition , 2014, IEEE Transactions on Intelligent Transportation Systems.

[88]  Pierre Charbonnier,et al.  Road Sign Detection in Images: A Case Study , 2010, 2010 20th International Conference on Pattern Recognition.

[89]  Ali Behloul,et al.  An overview of traffic sign detection and classification methods , 2017, International Journal of Multimedia Information Retrieval.

[90]  Wenyu Liu,et al.  Traffic sign detection and recognition using fully convolutional network guided proposals , 2016, Neurocomputing.

[91]  Sergio Lafuente-Arroyo,et al.  Fundaments in Luminance and Retroreflectivity Measurements of Vertical Traffic Signs Using a Color Digital Camera , 2008, IEEE Transactions on Instrumentation and Measurement.

[92]  Sinisa Segvic,et al.  A computer vision assisted geoinformation inventory for traffic infrastructure , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[93]  Luke Fletcher,et al.  Real-Time Speed Sign Detection Using the Radial Symmetry Detector , 2008, IEEE Transactions on Intelligent Transportation Systems.