Vision-Based Traffic Sign Detection and Recognition Systems: Current Trends and Challenges

The automatic traffic sign detection and recognition (TSDR) system is very important research in the development of advanced driver assistance systems (ADAS). Investigations on vision-based TSDR have received substantial interest in the research community, which is mainly motivated by three factors, which are detection, tracking and classification. During the last decade, a substantial number of techniques have been reported for TSDR. This paper provides a comprehensive survey on traffic sign detection, tracking and classification. The details of algorithms, methods and their specifications on detection, tracking and classification are investigated and summarized in the tables along with the corresponding key references. A comparative study on each section has been provided to evaluate the TSDR data, performance metrics and their availability. Current issues and challenges of the existing technologies are illustrated with brief suggestions and a discussion on the progress of driver assistance system research in the future. This review will hopefully lead to increasing efforts towards the development of future vision-based TSDR system.

[1]  José Manuel Pastor,et al.  Visual sign information extraction and identification by deformable models for intelligent vehicles , 2004, IEEE Transactions on Intelligent Transportation Systems.

[2]  Lan Lin,et al.  Traffic Signs Recognition and Classification based on Deep Feature Learning , 2018, ICPRAM.

[3]  H. Fleyeh.  Shadow And Highlight Invariant Colour Segmentation Algorithm For Traffic Signs , 2006, 2006 IEEE Conference on Cybernetics and Intelligent Systems.

[4]  Sancho Salcedo-Sanz,et al.  A decision support system for the automatic management of keep-clear signs based on support vector machines and geographic information systems , 2010, Expert Syst. Appl..

[5]  Faliang Chang,et al.  Supplemental Boosting and Cascaded ConvNet Based Transfer Learning Structure for Fast Traffic Sign Detection in Unknown Application Scenes , 2018, Sensors.

[6]  Majid Mirmehdi,et al.  Traffic sign recognition using MSER and Random Forests , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[7]  Zengfu Wang,et al.  Real-Time Traffic Sign Recognition Based on Efficient CNNs in the Wild , 2019, IEEE Transactions on Intelligent Transportation Systems.

[8]  Daniel Petrisor,et al.  Experimental traffic sign detection using I2V communication , 2012, 2012 35th International Conference on Telecommunications and Signal Processing (TSP).

[9]  Xiaohui Liu,et al.  Towards Real-Time Traffic Sign Recognition by Class-Specific Discriminative Features , 2007, BMVC.

[10]  Zhao Wang,et al.  Robust chinese traffic sign detection and recognition with deep convolutional neural network , 2015, 2015 11th International Conference on Natural Computation (ICNC).

[11]  S. Lafuente-Arroyo,et al.  A Tracking System for Automated Inventory of Road Signs , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[12]  Aref Meddeb,et al.  Deep learning traffic sign detection, recognition and augmentation , 2017, SAC.

[13]  Yen-Lin Chen,et al.  Road speed sign recognition using edge-voting principle and learning vector quantization network , 2010, 2010 International Computer Symposium (ICS2010).

[14]  Z. Kalafatic,et al.  Real-time detection and recognition of traffic signs , 2010, The 33rd International Convention MIPRO.

[15]  Le Zhang,et al.  Traffic-signs recognition system based on multi-features , 2012, 2012 IEEE International Conference on Computational Intelligence for Measurement Systems and Applications (CIMSA) Proceedings.

[16]  Bin Fan,et al.  Traffic Sign Recognition Using a Multi-Task Convolutional Neural Network , 2018, IEEE Transactions on Intelligent Transportation Systems.

[17]  Majid Mirmehdi,et al.  Real-Time Detection and Recognition of Road Traffic Signs , 2012, IEEE Transactions on Intelligent Transportation Systems.

[18]  Nilesh J. Uke.  DETECTION, CLASSIFICATION AND RECOGNITION OF ROAD TRAFFIC SIGNS USING COLOR AND SHAPE , 2012 .

[19]  Saturnino Maldonado-Bascón,et al.  Traffic sign shape classification based on correlation techniques , 2005 .

[20]  Shiliang Pu,et al.  Smart data driven traffic sign detection method based on adaptive color threshold and shape symmetry , 2019, Future Gener. Comput. Syst..

[21]  T. Asakura,et al.  A study on traffic sign recognition in scene image using genetic algorithms and neural networks , 1996, Proceedings of the 1996 IEEE IECON. 22nd International Conference on Industrial Electronics, Control, and Instrumentation.

[22]  Real-Time Traffic Sign Detection: An Evaluation Study , 2010, 2010 20th International Conference on Pattern Recognition.

[23]  Suk-Hwan Lee,et al.  Broken traffic sign recognition based on local histogram matching , 2012, 2012 Computing, Communications and Applications Conference.

[24]  Sei-Wang Chen,et al.  An automatic road sign recognition system based on a computational model of human recognition processing , 2004, Comput. Vis. Image Underst..

[25]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[26]  Cuneyt Akinlar,et al.  Circular traffic sign recognition empowered by circle detection algorithm , 2013, 2013 21st Signal Processing and Communications Applications Conference (SIU).

[27]  Mohammed A. Hannan,et al.  An Automatic Traffic Sign Detection and Recognition System Based on Colour Segmentation, Shape Matching, and SVM , 2015 .

[28]  O. Ghita,et al.  A robust algorithm for detection and classification of traffic signs in video data , 2012, 2012 International Conference on Control, Automation and Information Sciences (ICCAIS).

[29]  Michael Anthony Bauer,et al.  Detection and recognition of traffic signs inside the attentional visual field of drivers , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[30]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[31]  Vishal R. Deshmukh,et al.  Real-Time Traffic Sign Recognition System based on Colour Image Segmentation , 2013 .

[32]  Qingquan Li,et al.  Traffic sign detection and recognition for intelligent vehicle , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[33]  Cheng Liang,et al.  Automatic detection and recognition of traffic signs in stereo images based on features and probabilistic neural networks , 2008, SPIE Photonics Europe.

[34]  Meng-Yin Fu,et al.  A survey of traffic sign recognition , 2010, 2010 International Conference on Wavelet Analysis and Pattern Recognition.

[35]  Fatin Zaklouta,et al.  Real-time traffic sign recognition using spatially weighted HOG trees , 2011, 2011 15th International Conference on Advanced Robotics (ICAR).

[36]  Fatih Murat Porikli,et al.  Fast Detection of Multiple Objects in Traffic Scenes With a Common Detection Framework , 2015, IEEE Transactions on Intelligent Transportation Systems.

[37]  Ivan Cruz-Aceves,et al.  Eigen-Gradients for Traffic Sign Recognition , 2013 .

[38]  Giovanni Pilato,et al.  Road signs recognition using a dynamic pixel aggregation technique in the HSV color space , 2001, Proceedings 11th International Conference on Image Analysis and Processing.

[39]  Luke Fletcher,et al.  Real-Time Speed Sign Detection Using the Radial Symmetry Detector , 2008, IEEE Transactions on Intelligent Transportation Systems.

[40]  Dzuraidah Abd. Wahab,et al.  Decision fusion of a multi-sensing embedded system for occupant safety measures , 2010 .

[41]  Ahmet Sayar,et al.  CNN Based Traffic Sign Recognition for Mini Autonomous Vehicles , 2018, Advances in Intelligent Systems and Computing.

[42]  Cui-Hua Li,et al.  Unifying visual saliency with HOG feature learning for traffic sign detection , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[43]  Fatin Zaklouta,et al.  Real-time traffic sign recognition in three stages , 2014, Robotics Auton. Syst..

[44]  Jun Li,et al.  Robust traffic sign detection using fuzzy shape recognizer , 2009, International Symposium on Multispectral Image Processing and Pattern Recognition.

[45]  Jordi Vitrià,et al.  Traffic-Sign Recognition Systems , 2011, Springer Briefs in Computer Science.

[46]  Faliang Chang,et al.  Fast Traffic Sign Recognition via High-Contrast Region Extraction and Extended Sparse Representation , 2016, IEEE Transactions on Intelligent Transportation Systems.

[47]  Stefano Cagnoni,et al.  Real-Time GPU Based Road Sign Detection and Classification , 2012, PPSN.

[48]  Brian V. Funt,et al.  Color Constant Color Indexing , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Xiaohong W. Gao,et al.  Recognition of traffic signs based on their colour and shape features extracted using human vision models , 2006, J. Vis. Commun. Image Represent..

[50]  Johannes Stallkamp,et al.  The German Traffic Sign Recognition Benchmark: A multi-class classification competition , 2011, The 2011 International Joint Conference on Neural Networks.

[51]  S. Lafuente-Arroyo,et al.  Road Sign Tracking with a Predictive Filter Solution , 2006, IECON 2006 - 32nd Annual Conference on IEEE Industrial Electronics.

[52]  Qi Wang,et al.  An Incremental Framework for Video-Based Traffic Sign Detection, Tracking, and Recognition , 2017, IEEE Transactions on Intelligent Transportation Systems.

[53]  Dip Nandi,et al.  Traffic Sign Detection based on Color Segmentation of Obscure Image Candidates: A Comprehensive Study , 2018, International Journal of Modern Education and Computer Science.

[54]  Byong-Woo Kim,et al.  Traffic sign extract and recognition on unmanned vehicle using image processing based on support vector machine , 2011, 2011 11th International Conference on Control, Automation and Systems.

[55]  Ralph R. Martin,et al.  An Optimization Approach for Localization Refinement of Candidate Traffic Signs , 2017, IEEE Transactions on Intelligent Transportation Systems.

[56]  Jesmin F. Khan,et al.  Image Segmentation and Shape Analysis for Road-Sign Detection , 2011, IEEE Transactions on Intelligent Transportation Systems.

[57]  Bin Ran,et al.  Vision-Based Stop Sign Detection and Recognition System for Intelligent Vehicles , 2001 .

[58]  Amara Dinesh Kumar,et al.  Novel Deep Learning Model for Traffic Sign Detection Using Capsule Networks , 2018, ArXiv.

[59]  L. Felipe Blázquez,et al.  Traffic Sign Recognition Application Based on Image Processing Techniques , 2014 .

[60]  J. Torresen,et al.  Efficient recognition of speed limit signs , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[61]  Yi Yang,et al.  Semantic Pooling for Complex Event Analysis in Untrimmed Videos , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Rubiyah Yusof,et al.  Malaysian traffic sign dataset for traffic sign detection and recognition systems , 2016 .

[63]  Xiaojun Chang,et al.  Semisupervised Feature Analysis by Mining Correlations Among Multiple Tasks , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[64]  Kyung-Joong Kim,et al.  Design of a visual perception model with edge-adaptive Gabor filter and support vector machine for traffic sign detection , 2013, Expert Syst. Appl..

[65]  Qingquan Li,et al.  Design of a Multi-Sensor Cooperation Travel Environment Perception System for Autonomous Vehicle , 2012, Sensors.

[66]  Kang Ryoung Park,et al.  Recognition of Damaged Arrow-Road Markings by Visible Light Camera Sensor Based on Convolutional Neural Network , 2016, Sensors.

[67]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[68]  Zhilu Wu,et al.  A robust, coarse-to-fine traffic sign detection method , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[69]  Iping Supriana,et al.  Traffic sign recognition with Color-based Method, shape-arc estimation and SVM , 2011, Proceedings of the 2011 International Conference on Electrical Engineering and Informatics.

[70]  S. Xu,et al.  Robust traffic sign shape recognition using geometric matching , 2009 .

[71]  J. M. Armingol,et al.  TRAFFIC SIGN DETECTION FOR DRIVER SUPPORT SYSTEMS , 2022 .

[72]  Toshiaki Fujii,et al.  Traffic sign detection in dual-focal active camera system , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[73]  Lutz Priese,et al.  On hierarchical color segmentation and applications , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[74]  Antonio Fernández-Caballero,et al.  An optimization on pictogram identification for the road-sign recognition task using SVMs , 2010, Comput. Vis. Image Underst..

[75]  A. Mtibaa,et al.  Automatic detection and recognition of road sign for driver assistance system , 2012, 2012 16th IEEE Mediterranean Electrotechnical Conference.

[76]  Jianping Fan,et al.  Automatic image segmentation by integrating color-edge extraction and seeded region growing , 2001, IEEE Trans. Image Process..

[77]  Arturo de la Escalera,et al.  Recognition Stage for a Speed Supervisor Based on Road Sign Detection , 2012, Sensors.

[78]  D. B. Megherbi,et al.  Automatic detection and recognition of traffic road signs for intelligent autonomous unmanned vehicles for urban surveillance and rescue , 2010, 2010 IEEE International Conference on Technologies for Homeland Security (HST).

[79]  Klaus Zimmermann,et al.  Towards reliable traffic sign recognition , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[80]  Xiaohui Liu,et al.  Detection, Tracking and Recognition of Traffic Signs from Video Input , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[81]  Fuchun Sun,et al.  Supervised Low-Rank Matrix Recovery for Traffic Sign Recognition in Image Sequences , 2013, IEEE Signal Processing Letters.

[82]  N. Pettersson,et al.  The histogram feature - a resource-efficient Weak Classifier , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[83]  Reinhard Klette,et al.  General traffic sign recognition by feature matching , 2009, 2009 24th International Conference Image and Vision Computing New Zealand.

[84]  Fang-Hsuan Cheng,et al.  A Method of Detecting and Recognizing Speed-limit Signs , 2012, 2012 Eighth International Conference on Intelligent Information Hiding and Multimedia Signal Processing.

[85]  Yang Siyan,et al.  Road-sign segmentation and recognition in natural scenes , 2011, 2011 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC).

[86]  David Fernández Llorca,et al.  Robust traffic signs detection by means of vision and V2I communications , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[87]  Dariu Gavrila,et al.  Traffic Sign Recognition Revisited , 1999, DAGM-Symposium.

[88]  Francisco López-Ferreras,et al.  Road-Sign Detection and Recognition Based on Support Vector Machines , 2007, IEEE Transactions on Intelligent Transportation Systems.

[89]  Domenec Puig,et al.  A practical approach for detection and classification of traffic signs using Convolutional Neural Networks , 2016, Robotics Auton. Syst..

[90]  Peter H. N. de With,et al.  Color exploitation in hog-based traffic sign detection , 2010, 2010 IEEE International Conference on Image Processing.

[91]  Azah Mohamed,et al.  A Unified Robust Algorithm for Detection of Human and Non-human Object in Intelligent Safety Application , 2008 .

[92]  Johannes Stallkamp,et al.  Detection of traffic signs in real-world images: The German traffic sign detection benchmark , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[93]  Ikuko Nishikawa,et al.  Detection and recognition of road signs using simple layered neural networks , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[94]  Pranjal Protim Borah,et al.  A Study on Automatic Detection and Recognition Techniques For Road Signs , 2015 .

[95]  Dong Wang,et al.  Real-Time Traffic Sign Detection and Recognition Method Based on Simplified Gabor Wavelets and CNNs , 2018, Sensors.

[96]  Visvanathan Ramesh,et al.  A system for traffic sign detection, tracking, and recognition using color, shape, and motion information , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[97]  Donghoon Yeo,et al.  New Dark Area Sensitive Tone Mapping for Deep Learning Based Traffic Sign Recognition , 2018, Sensors.

[98]  Arturo de la Escalera,et al.  Traffic sign recognition and analysis for intelligent vehicles , 2003, Image Vis. Comput..

[99]  Alastair R. Allen,et al.  Using self-organising maps in the detection and recognition of road signs , 2009, Image Vis. Comput..

[100]  Yan Jiang,et al.  Traffic sign recognition using Ridge Regression and OTSU method , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[101]  Xiaohui Liu,et al.  Real-time traffic sign recognition from video by class-specific discriminative features , 2010, Pattern Recognit..

[102]  Yanyun Qu,et al.  Traffic-Sign Spotting in the Wild via Deep Features , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[103]  Fatin Zaklouta,et al.  Warning traffic sign recognition using a HOG-based K-d tree , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[104]  Ram Gopal Raj,et al.  Real-Time (Vision-Based) Road Sign Recognition Using an Artificial Neural Network , 2017, Sensors.

[105]  Yunchao Wei,et al.  Perceptual Generative Adversarial Networks for Small Object Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[106]  Ishpreet Singh,et al.  Comparison between Edge Detection Techniques , 2016 .

[107]  Sebastian Houben,et al.  A single target voting scheme for traffic sign detection , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[108]  Fatin Zaklouta,et al.  Real-Time Traffic-Sign Recognition Using Tree Classifiers , 2012, IEEE Transactions on Intelligent Transportation Systems.

[109]  Francisco López-Ferreras,et al.  Traffic sign shape classification and localization based on the normalized FFT of the signature of blobs and 2D homographies , 2008, Signal Processing.

[110]  Kang Ryoung Park,et al.  Road Lane Detection Robust to Shadows Based on a Fuzzy System Using a Visible Light Camera Sensor , 2017, Sensors.

[111]  Bruno Vallet,et al.  Detection and 3D reconstruction of traffic signs from multiple view color images , 2013 .

[112]  Ali Behloul,et al.  Speed limit sign detection and recognition system using SVM and MNIST datasets , 2019, Neural Computing and Applications.

[113]  Jürgen Schmidhuber,et al.  Multi-column deep neural network for traffic sign classification , 2012, Neural Networks.

[114]  Hiroshi Murase,et al.  Intelligent traffic sign detector: Adaptive learning based on online gathering of training samples , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[115]  Jenn-Jier James Lien,et al.  Road sign recognition system based on GentleBoost with sharing features , 2011, Proceedings 2011 International Conference on System Science and Engineering.

[116]  Yann LeCun,et al.  Traffic sign recognition with multi-scale Convolutional Networks , 2011, The 2011 International Joint Conference on Neural Networks.

[117]  Joon Hee Han,et al.  Image indexing using weighted color histogram , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[118]  Ju H. Park,et al.  An Efficient Color Space for Deep-Learning Based Traffic Light Recognition , 2018, Journal of Advanced Transportation.

[119]  Ming-Shi Wang,et al.  Road Sign Recognition with Fuzzy Adaptive Pre-Processing Models , 2012, Sensors.

[120]  João Paulo Papa,et al.  Embedded real-time speed limit sign recognition using image processing and machine learning techniques , 2016, Neural Computing and Applications.

[121]  Lars Petersson,et al.  Large scale sign detection using HOG feature variants , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[122]  Changshui Zhang,et al.  Traffic Sign Recognition With Hinge Loss Trained Convolutional Neural Networks , 2014, IEEE Transactions on Intelligent Transportation Systems.

[123]  Kang Kim,et al.  Simultaneous Traffic Sign Detection and Boundary Estimation Using Convolutional Neural Network , 2018, IEEE Transactions on Intelligent Transportation Systems.

[124]  Hyeran Byun,et al.  Real-time traffic sign recognition based on a general purpose GPU and deep-learning , 2017, PloS one.

[125]  Reza A. Zoroofi,et al.  Gabor wavelet for road sign detection and recognition using a hybrid classifier , 2010, 2010 International Conference on Multimedia Computing and Information Technology (MCIT).

[126]  Thomas B. Moeslund,et al.  Vision-Based Traffic Sign Detection and Analysis for Intelligent Driver Assistance Systems: Perspectives and Survey , 2012, IEEE Transactions on Intelligent Transportation Systems.

[127]  Abdulrahman S. Alturki.  Traffic Sign Detection and Recognition Using Adaptive Threshold Segmentation with Fuzzy Neural Network Classification , 2018, 2018 International Symposium on Networks, Computers and Communications (ISNCC).

[128]  Ma Guang-fu.  Detection and classification of traffic signs in natural environments , 2009 .

[129]  Aini Hussain,et al.  Decision fusion via integrated sensing system for a smart airbag deployment scheme , 2011 .

[130]  B. Philip,et al.  A Road Traffic Signal Recognition System Based on Template Matching Employing Tree Classifier , 2007, International Conference on Computational Intelligence and Multimedia Applications (ICCIMA 2007).

[131]  Gareth Blake Loy,et al.  Fast shape-based road sign detection for a driver assistance system , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[132]  Ali Behloul,et al.  An overview of traffic sign detection and classification methods , 2017, International Journal of Multimedia Information Retrieval.

[133]  Fatin Zaklouta,et al.  Segmentation masks for real-time traffic sign recognition using weighted HOG-based trees , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[134]  Luc Van Gool,et al.  Integrating Object Detection with 3D Tracking Towards a Better Driver Assistance System , 2010, 2010 20th International Conference on Pattern Recognition.

[135]  Yike Guo,et al.  Fast Traffic Sign Recognition with a Rotation Invariant Binary Pattern Based Feature , 2015, Sensors.

[136]  A. Broggi,et al.  Real Time Road Signs Recognition , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[137]  Liu Yang,et al.  Real-Time Detection and Recognition of Road Traffic Signs using MSER and Random Forests , 2018, Int. J. Online Eng..

[138]  Wenyu Liu,et al.  Cascaded Segmentation-Detection Networks for Text-Based Traffic Sign Detection , 2018, IEEE Transactions on Intelligent Transportation Systems.

[139]  Jürgen Schmidhuber,et al.  A committee of neural networks for traffic sign classification , 2011, The 2011 International Joint Conference on Neural Networks.

[140]  Koichi Yamada,et al.  Improving the performance of traffic sign detection using blob tracking , 2007, IEICE Electron. Express.

[141]  Sei-Wang Chen,et al.  Road-sign detection and tracking , 2003, IEEE Trans. Veh. Technol..

[142]  Sven Loncaric,et al.  Traffic sign detection and recognition using neural networks and histogram based selection of segmentation method , 2011, Proceedings ELMAR-2011.

[143]  Saturnino Maldonado-Bascón,et al.  Goal Evaluation of Segmentation Algorithms for Traffic Sign Recognition , 2010, IEEE Transactions on Intelligent Transportation Systems.

[144]  Changjun Li,et al.  The Performance of CIECAM02 , 2002, Color Imaging Conference.

[145]  Hong-Bin Ma,et al.  A Combined Method for Traffic Sign Detection and Classification , 2010, 2010 Chinese Conference on Pattern Recognition (CCPR).