Real-Time (Vision-Based) Road Sign Recognition Using an Artificial Neural Network

Road sign recognition is a driver support function that can be used to notify and warn the driver by showing the restrictions that may be effective on the current stretch of road. Examples for such regulations are ‘traffic light ahead’ or ‘pedestrian crossing’ indications. The present investigation targets the recognition of Malaysian road and traffic signs in real-time. Real-time video is taken by a digital camera from a moving vehicle and real world road signs are then extracted using vision-only information. The system is based on two stages, one performs the detection and another one is for recognition. In the first stage, a hybrid color segmentation algorithm has been developed and tested. In the second stage, an introduced robust custom feature extraction method is used for the first time in a road sign recognition approach. Finally, a multilayer artificial neural network (ANN) has been created to recognize and interpret various road signs. It is robust because it has been tested on both standard and non-standard road signs with significant recognition accuracy. This proposed system achieved an average of 99.90% accuracy with 99.90% of sensitivity, 99.90% of specificity, 99.90% of f-measure, and 0.001 of false positive rate (FPR) with 0.3 s computational time. This low FPR can increase the system stability and dependability in real-time applications.

[1]  Johannes Stallkamp,et al.  Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition , 2012, Neural Networks.

[2]  Luis Moreno,et al.  Road traffic sign detection and classification , 1997, IEEE Trans. Ind. Electron..

[3]  Ata M. Khan.  Design of Real-Time Transition from Driving Assistance to Automation Function: Bayesian Artificial Intelligence Approach , 2014 .

[4]  Alpha Agape Gopalai,et al.  Malaysia traffic sign recognition with convolutional neural network , 2015, 2015 IEEE International Conference on Digital Signal Processing (DSP).

[5]  Thomas B. Moeslund,et al.  Vision-Based Traffic Sign Detection and Analysis for Intelligent Driver Assistance Systems: Perspectives and Survey , 2012, IEEE Transactions on Intelligent Transportation Systems.

[6]  David Fernández Llorca,et al.  Vision-Based Traffic Data Collection Sensor for Automotive Applications , 2010, Sensors.

[7]  Yichang James Tsai,et al.  Geometry Preserving Active Polygon-Incorporated Sign Detection Algorithm , 2015 .

[8]  Yichang James Tsai,et al.  Hybrid Active Contour-Incorporated Sign Detection Algorithm , 2012, J. Comput. Civ. Eng..

[9]  Shahryar Rahnamayan,et al.  Fuzzy Adaptive Cruise Control system with speed sign detection capability , 2014, 2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[10]  Mani Golparvar-Fard,et al.  Detection, classification, and mapping of U.S. traffic signs using google street view images for roadway inventory management , 2015 .

[11]  Dia I. Abu-Al-Nadi,et al.  Road traffic sign detection in color images , 2003, 10th IEEE International Conference on Electronics, Circuits and Systems, 2003. ICECS 2003. Proceedings of the 2003.

[12]  Azzedine Boukerche,et al.  Road-Sign Text Recognition Architecture for Intelligent Transportation Systems , 2014, 2014 IEEE 80th Vehicular Technology Conference (VTC2014-Fall).

[13]  Eiji Hayashi.  A Navigation System with a self-drive control for an Autonomous Robot in an Indoor Environment , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[14]  Majid Mirmehdi,et al.  Recognizing Text-Based Traffic Signs , 2015, IEEE Transactions on Intelligent Transportation Systems.

[15]  Sei-ichiro Kamata,et al.  Improved Color Barycenter Model and Its Separation for Road Sign Detection , 2013, IEICE Trans. Inf. Syst..

[16]  Hasan Fleyeh,et al.  Detection and classification of speed limit traffic signs , 2014, 2014 World Congress on Computer Applications and Information Systems (WCCAIS).

[17]  Tao Chen,et al.  Accurate and Efficient Traffic Sign Detection Using Discriminative AdaBoost and Support Vector Regression , 2016, IEEE Transactions on Vehicular Technology.

[18]  David Fernández Llorca,et al.  Complete Vision-Based Traffic Sign Recognition Supported by an I2V Communication System , 2012, Sensors.

[19]  Saturnino Maldonado-Bascón,et al.  Goal Evaluation of Segmentation Algorithms for Traffic Sign Recognition , 2010, IEEE Transactions on Intelligent Transportation Systems.

[20]  José Manuel Pastor,et al.  Visual sign information extraction and identification by deformable models for intelligent vehicles , 2004, IEEE Transactions on Intelligent Transportation Systems.

[21]  H. Fleyeh,et al.  Color detection and segmentation for road and traffic signs , 2004, IEEE Conference on Cybernetics and Intelligent Systems, 2004..

[22]  S. Lafuente-Arroyo,et al.  Traffic sign shape classification based on Support Vector Machines and the FFT of the signature of blobs , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[23]  Kang Ryoung Park,et al.  Road Lane Detection by Discriminating Dashed and Solid Road Lanes Using a Visible Light Camera Sensor , 2016, Sensors.

[24]  Xiaohui Liu,et al.  Real-time traffic sign recognition from video by class-specific discriminative features , 2010, Pattern Recognit..

[25]  Rajeev Kumaraswamy,et al.  SVM Based Classification of Traffic Signs for Realtime Embedded Platform , 2011, ACC.

[26]  Naveen Shivaramu Yeshodara,et al.  Cloud Based Self Driving Cars , 2014, 2014 IEEE International Conference on Cloud Computing in Emerging Markets (CCEM).

[27]  Yann LeCun,et al.  Traffic sign recognition with multi-scale Convolutional Networks , 2011, The 2011 International Joint Conference on Neural Networks.

[28]  Daniel Petrisor,et al.  Detection and recognition of traffic signs using Gabor filters , 2011, 2011 34th International Conference on Telecommunications and Signal Processing (TSP).

[29]  Ming-Shi Wang,et al.  Road Sign Recognition with Fuzzy Adaptive Pre-Processing Models , 2012, Sensors.

[30]  Hassan Basri,et al.  Real time road sign recognition system using artificial neural networks for bengali textual information box , 2008, 2008 International Symposium on Information Technology.

[31]  Nursabilillah Mohd Ali,et al.  Individual processing speed analysis for traffic sign detection and recognition , 2013, 2013 IEEE International Conference on Smart Instrumentation, Measurement and Applications (ICSIMA).

[32]  Jian Wang,et al.  Road sign detection using specific color-pair information , 2012, 2012 International Conference on Machine Learning and Cybernetics.

[33]  Nourdine Aliane,et al.  A System for Traffic Violation Detection , 2014, Sensors.

[34]  Yike Guo,et al.  Fast Traffic Sign Recognition with a Rotation Invariant Binary Pattern Based Feature , 2015, Sensors.

[35]  David Fernández Llorca,et al.  Robust traffic signs detection by means of vision and V2I communications , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[36]  Francisco López-Ferreras,et al.  Road-Sign Detection and Recognition Based on Support Vector Machines , 2007, IEEE Transactions on Intelligent Transportation Systems.

[37]  Tao Zhang,et al.  Road sign detection based on visual saliency and shape analysis , 2013, 2013 IEEE International Conference on Image Processing.

[38]  Kah Phooi Seng,et al.  MIMO Lyapunov Theory-Based RBF Neural Classifier for Traffic Sign Recognition , 2012, Appl. Comput. Intell. Soft Comput..

[39]  Johannes Stallkamp,et al.  Detection of traffic signs in real-world images: The German traffic sign detection benchmark , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[40]  Mohammad Badrul Alam Miah,et al.  A Real Time Road Sign Recognition using Neural Network , 2015 .

[41]  Sergio Saponara,et al.  Real-time color/shape-based traffic signs acquisition and recognition system , 2013, Electronic Imaging.

[42]  Kyoung-Wook Min,et al.  Design and implementation of an intelligent vehicle system for autonomous valet parking service , 2015, 2015 10th Asian Control Conference (ASCC).

[43]  Frank P. Ferrie,et al.  A model-based road sign identification system , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[44]  Jürgen Schmidhuber,et al.  Multi-column deep neural network for traffic sign classification , 2012, Neural Networks.

[45]  A. Suruliandi,et al.  Performance analysis on road sign detection, extraction and recognition techniques , 2013, 2013 International Conference on Circuits, Power and Computing Technologies (ICCPCT).

[46]  Jianping Wu,et al.  Real-Time Automatic Road Sign Detection , 2009, 2009 Fifth International Conference on Image and Graphics.

[47]  Majid Mirmehdi,et al.  Real-Time Detection and Recognition of Road Traffic Signs , 2012, IEEE Transactions on Intelligent Transportation Systems.

[48]  Jesmin F. Khan,et al.  Image Segmentation and Shape Analysis for Road-Sign Detection , 2011, IEEE Transactions on Intelligent Transportation Systems.

[49]  J. B. Kim.  Detection of traffic signs based on eigen-color model and saliency model in driver assistance systems , 2013 .

[50]  Pierre Charbonnier,et al.  Road Sign Detection in Images: A Case Study , 2010, 2010 20th International Conference on Pattern Recognition.

[51]  Kaushik Deb,et al.  Bangladeshi road sign detection based on YCbCr color model and DtBs vector , 2015, 2015 International Conference on Computer and Information Engineering (ICCIE).

[52]  Kang-Hyun Jo,et al.  Information retrieval of LED text on electronic road sign for driver-assistance system using spatial-based feature and Nearest Cluster Neighbor classifier , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[53]  Kenji Terada,et al.  Japanese road signs recognition using neural networks , 2013, The SICE Annual Conference 2013.

[54]  Arturo de la Escalera,et al.  Recognition Stage for a Speed Supervisor Based on Road Sign Detection , 2012, Sensors.

[55]  Chokri Abdelmoula,et al.  A Novel Neural Network Based Method Developed for Digit Recognition Applied to Automatic Speed Sign Recognition , 2016 .

[56]  Jiun-In Guo,et al.  Algorithm derivation and its embedded system realization of speed limit detection for multiple countries , 2016, 2016 IEEE International Symposium on Circuits and Systems (ISCAS).

[57]  Qingquan Li,et al.  Design of a Multi-Sensor Cooperation Travel Environment Perception System for Autonomous Vehicle , 2012, Sensors.

[58]  Mohammed A. Hannan,et al.  An Automatic Traffic Sign Detection and Recognition System Based on Colour Segmentation, Shape Matching, and SVM , 2015 .

[59]  O. Ghita,et al.  A robust algorithm for detection and classification of traffic signs in video data , 2012, 2012 International Conference on Control, Automation and Information Sciences (ICCAIS).

[60]  S. Dalela,et al.  Automatic colored traffic sign detection using optoelectronic correlation architectures , 2008, 2008 IEEE International Conference on Vehicular Electronics and Safety.

[61]  Lihui Zhang,et al.  Optimize the Settings of Variable Speed Limit System to Improve the Performance of Freeway Traffic , 2015, IEEE Transactions on Intelligent Transportation Systems.

[62]  Yang Hao,et al.  Detecting Vital Signs with Wearable Wireless Sensors , 2010, Sensors.

[63]  J. Khurshid,et al.  Road Sign Detection and Recognition using Colour Segmentation, Shape Analysis and Template Matching , 2007, 2007 International Conference on Machine Learning and Cybernetics.

[64]  Robert P. W. Duin,et al.  Building Road-Sign Classifiers Using a Trainable Similarity Measure , 2006, IEEE Transactions on Intelligent Transportation Systems.

[65]  Sei-ichiro Kamata,et al.  Pixel color feature enhancement for road signs detection , 2010, International Conference on Digital Image Processing.

[66]  Yunju Baek,et al.  Design and Implementation of Real-Time Vehicular Camera for Driver Assistance and Traffic Congestion Estimation , 2015, Sensors.

[67]  Vishal R. Deshmukh,et al.  Real-Time Traffic Sign Recognition System based on Colour Image Segmentation , 2013 .

[68]  Mohan M. Trivedi,et al.  Pushing the “Speed Limit”: High-Accuracy US Traffic Sign Recognition With Convolutional Neural Networks , 2016, IEEE Transactions on Intelligent Vehicles.

[69]  Mani Golparvar-Fard,et al.  Video-Based Detection and Classification of US Traffic Signs and Mile Markers using Color Candidate Extraction and Feature-Based Recognition , 2014 .

[70]  Md. Al-Amin Bhuiyan,et al.  Neural Network based Road Sign Recognition , 2012 .

[71]  Jürgen Schmidhuber,et al.  A committee of neural networks for traffic sign classification , 2011, The 2011 International Joint Conference on Neural Networks.

[72]  Auranuch Lorsakul,et al.  Traffic Sign Recognition for Intelligent Vehicle/Driver Assistance System Using Neural Network on OpenCV , 2007 .

[73]  Seongdo Kim,et al.  Color and shape feature-based detection of speed sign in real-time , 2012, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[74]  H. Fleyeh.  Shadow And Highlight Invariant Colour Segmentation Algorithm For Traffic Signs , 2006, 2006 IEEE Conference on Cybernetics and Intelligent Systems.

[75]  Lei Song,et al.  Color-based traffic sign detection , 2012, 2012 International Conference on Quality, Reliability, Risk, Maintenance, and Safety Engineering.

[76]  Kalyan Goswami,et al.  Hybrid color space based road sign detection technique , 2012, 2012 IEEE Second International Conference on Consumer Electronics - Berlin (ICCE-Berlin).