Development of a Real-Time Traffic Sign Recognition System Based On Deep Learning Approach with Convolutional Neural Networks and Integrating to The Embedded Systems

Traffic signs are mandatory features of road traffic regulations worldwide. Automatic detection and recognition of traffic signs by vehicles may increase the safety level of drivers and passengers. For this reason, Real Time-Traffic Sign Recognition (RT-TSR) system is one of the essential components for smart transportation systems and high-tech vehicles. Recently, very good performances have been achieved in public datasets, especially with advanced Computer Vision (CV) approaches like Deep Learning (DL). Nevertheless, these CV techniques still need to be improved to provide the requirements of Real-Time (RT) applications. Although hopefully outcomes have been obtained theoretically in previous Traffic Sign Recognition (TSR) studies, there are very few studies that offer RT solutions in the real world. Therefore, in this study, a DL-based RT-TSR system is developed because of its high rate of recognition and quick execution. Besides, the CV approach has been included in the software developed to achieve classification as RT and support digital imaging. This developed system is capable of running smoothly in embedded systems with mobile GPU or CPU thanks to its low computational cost and high performance. Therefore, this study makes two important contributions to this field: software and hardware. First, RT-TSR software has been developed by using Convolutional Neural Networks (CNN) built on DL techniques along with CV techniques. Secondly, the developed software is adapted to embedded devices and hardware design is made. This developed system is also a technology product that offers software and hardware solutions together. Coding is accomplished under TensorFlow and OpenCV frameworks with the python programming language and CNN training is carried out by using parallel architecture. The experimental findings indicate that the developed CNN architecture achieves 99,71% accuracy and confirms the high efficiency of the system.

[1]  Xiaolin Hu,et al.  Traffic sign detection by ROI extraction and histogram features-based recognition , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[2]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[3]  Thomas B. Moeslund,et al.  Vision-Based Traffic Sign Detection and Analysis for Intelligent Driver Assistance Systems: Perspectives and Survey , 2012, IEEE Transactions on Intelligent Transportation Systems.

[4]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[5]  Cristina Olaverri-Monreal,et al.  Road Safety: Human Factors Aspects of Intelligent Vehicle Technologies , 2017, SMARTGREENS/VEHITS.

[6]  Sergey Andreev,et al.  Dense Moving Fog for Intelligent IoT: Key Challenges and Opportunities , 2018, IEEE Communications Magazine.

[7]  Yann LeCun,et al.  What is the best multi-stage architecture for object recognition? , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[8]  Jaafar Alghazo,et al.  Autonomous Traffic Sign (ATSR) Detection and Recognition using Deep CNN , 2019, Procedia Computer Science.

[9]  Eero P. Simoncelli,et al.  Nonlinear image representation using divisive normalization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Jürgen Schmidhuber,et al.  Multi-column deep neural network for traffic sign classification , 2012, Neural Networks.

[11]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[12]  Pierre Charbonnier,et al.  Road Sign Detection in Images: A Case Study , 2010, 2010 20th International Conference on Pattern Recognition.

[13]  Salina Abdul Samad,et al.  Comparative Survey on Traffic Sign Detection and Recognition: a Review , 2015 .

[14]  P. Yakimov,et al.  CNN Design for Real-Time Traffic Sign Recognition , 2017 .

[15]  Mrinal Haloi,et al.  Traffic Sign Classification Using Deep Inception Based Convolutional Networks , 2015, ArXiv.

[16]  Johannes Stallkamp,et al.  Detection of traffic signs in real-world images: The German traffic sign detection benchmark , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[17]  Arno Eichberger,et al.  Review of Recent Patents in Integrated Vehicle Safety, Advanced Driver Assistance Systems and Intelligent Transportation Systems , 2010 .

[18]  Lan Lin,et al.  Traffic Signs Recognition and Classification based on Deep Feature Learning , 2018, ICPRAM.

[19]  Shane Park,et al.  Systematic process to determine DNBR limit of CHF correlation with repetitive cross-validation technique , 2018 .

[20]  Fu Xiong,et al.  Towards Good Practices on Building Effective CNN Baseline Model for Person Re-identification , 2018, ArXiv.

[21]  Elliot Meyerson,et al.  Evolving Deep Neural Networks , 2017, Artificial Intelligence in the Age of Neural Networks and Brain Computing.

[22]  D. N. F. Awang Iskandar,et al.  Deep CNN based MR image denoising for tumor segmentation using watershed transform , 2018 .

[23]  Alexander Shustanov,et al.  A Method for Traffic Sign Recognition with CNN using GPU , 2017, SIGMAP.

[24]  Yann LeCun,et al.  Traffic sign recognition with multi-scale Convolutional Networks , 2011, The 2011 International Joint Conference on Neural Networks.

[25]  Chin-Chen Chang,et al.  Traffic Sign Detection and Recognition for Driving Assistance System , 2018 .

[26]  Yi Luo,et al.  Overview Research of Influence of In-Vehicle Intelligent Terminals on Drivers’ Distraction and Driving Safety , 2018 .

[27]  Jürgen Schmidhuber,et al.  A committee of neural networks for traffic sign classification , 2011, The 2011 International Joint Conference on Neural Networks.

[28]  Changshui Zhang,et al.  Traffic Sign Recognition With Hinge Loss Trained Convolutional Neural Networks , 2014, IEEE Transactions on Intelligent Transportation Systems.

[29]  Aref Meddeb,et al.  Deep learning traffic sign detection, recognition and augmentation , 2017, SAC.

[30]  George W. Irwin,et al.  A review on improving the autonomy of unmanned surface vehicles through intelligent collision avoidance manoeuvres , 2012, Annu. Rev. Control..

[31]  Feng Xiao,et al.  Improved Traffic Sign Detection and Recognition Algorithm for Intelligent Vehicles , 2019, Sensors.

[32]  Jean Ponce,et al.  A Theoretical Analysis of Feature Pooling in Visual Recognition , 2010, ICML.

[33]  Ghazanfar Latif,et al.  Deep Convolutional Neural Network for Recognition of Unified Multi-Language Handwritten Numerals , 2018, 2018 IEEE 2nd International Workshop on Arabic and Derived Script Analysis and Recognition (ASAR).