DeepThin: A novel lightweight CNN architecture for traffic sign recognition without GPU requirements

Abstract For a safe and automated vehicle driving application, it is a prerequisite to have a robust and highly accurate traffic sign detection system. In this paper, we proposed a novel energy-efficient Thin yet Deep convolutional neural network architecture for traffic sign recognition. Within the proposed architecture, each convolutional layer contains less than 50 features enabling our convolutional neural network to be trained quickly even without the aid of a graphics processing unit. The performance of the proposed architecture is measured using two publicly available traffic sign datasets, namely the German Traffic Sign Recognition Benchmark and the Belgian Traffic Sign Classification dataset. First, we train and test the performance of the proposed architecture using the large German Traffic Sign Recognition Benchmark dataset. Then, we retrain the network models using transfer learning on the more challenging Belgian Traffic Sign Classification dataset to evaluate test performance. The proposed architecture outperforms the performance of the state-of-the-art traffic sign methods with at least five times less parameter in the individual end-to-end network for training.

[1]  Pablo M. Granitto,et al.  Neural network ensembles: evaluation of aggregation algorithms , 2005, Artif. Intell..

[2]  A. Broggi,et al.  Real Time Road Signs Recognition , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[3]  Craig Webster,et al.  Robotics, Artificial Intelligence, and the Evolving Nature of Work , 2019, Digital Transformation in Business and Society.

[4]  Debasis Sarkar,et al.  Traffic Sign Detection and Recognition using a CNN Ensemble , 2019, 2019 IEEE International Conference on Consumer Electronics (ICCE).

[5]  StallkampJ.,et al.  2012 Special Issue , 2012 .

[6]  Siddharth Swarup Rautaray,et al.  Application of Deep Learning for Object Detection , 2018 .

[7]  Xiaojun Jing,et al.  Frequency Selective Convolutional Neural Networks for Traffic Sign Recognition , 2016, 2016 IEEE 83rd Vehicular Technology Conference (VTC Spring).

[8]  Cheng Wang,et al.  Bag-of-visual-phrases and hierarchical deep models for traffic sign detection and recognition in mobile laser scanning data , 2016 .

[9]  Anjan Gudigar,et al.  Multiple thresholding and subspace based approach for detection and recognition of traffic sign , 2017, Multimedia Tools and Applications.

[10]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[11]  Dayong Shen,et al.  Traffic Sign Recognition Using Kernel Extreme Learning Machines With Deep Perceptual Features , 2017, IEEE Transactions on Intelligent Transportation Systems.

[12]  Visvanathan Ramesh,et al.  A system for traffic sign detection, tracking, and recognition using color, shape, and motion information , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[13]  Cui-Hua Li,et al.  Unifying visual saliency with HOG feature learning for traffic sign detection , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[14]  Hakil Kim,et al.  Data debiased traffic sign recognition using MSERs and CNN , 2016, 2016 International Conference on Electronics, Information, and Communications (ICEIC).

[15]  Jürgen Schmidhuber,et al.  A committee of neural networks for traffic sign classification , 2011, The 2011 International Joint Conference on Neural Networks.

[16]  Jason Jianjun Gu,et al.  An Efficient Method for Traffic Sign Recognition Based on Extreme Learning Machine , 2017, IEEE Transactions on Cybernetics.

[17]  Johannes Stallkamp,et al.  Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition , 2012, Neural Networks.

[18]  Andrew P. Smith,et al.  A UK survey of driving behaviour, fatigue, risk taking and road traffic accidents , 2016, BMJ Open.

[19]  Gary Marcus,et al.  Deep Learning: A Critical Appraisal , 2018, ArXiv.

[20]  Alberto Ferreira de Souza,et al.  Self-Driving Cars: A Survey , 2019, Expert Syst. Appl..

[21]  Yann LeCun,et al.  Traffic sign recognition with multi-scale Convolutional Networks , 2011, The 2011 International Joint Conference on Neural Networks.

[22]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[23]  Yujun Zeng,et al.  Traffic Sign Recognition Using Extreme Learning Classifier with Deep Convolutional Features , 2015 .

[24]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[25]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[26]  Faliang Chang,et al.  Fast Traffic Sign Recognition via High-Contrast Region Extraction and Extended Sparse Representation , 2016, IEEE Transactions on Intelligent Transportation Systems.

[27]  Xiaohong W. Gao,et al.  Recognition of traffic signs based on their colour and shape features extracted using human vision models , 2006, J. Vis. Commun. Image Represent..

[28]  Max Tegmark,et al.  The role of artificial intelligence in achieving the Sustainable Development Goals , 2019, Nature Communications.

[29]  S. Lafuente-Arroyo,et al.  Traffic sign shape classification evaluation. Part II. FFT applied to the signature of blobs , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[30]  TRAFFIC SIGN RECOGNITION USING SCALE INVARIANT FEATURE TRANSFORM AND SVM , 2010 .

[31]  Jin Wang,et al.  Lightweight deep network for traffic sign classification , 2019, Annals of Telecommunications.

[32]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[33]  Jean-Luc Dugelay,et al.  Learned vs. Hand-Crafted Features for Pedestrian Gender Recognition , 2015, ACM Multimedia.

[34]  Xiao Lu,et al.  Traffic Sign Recognition via Multi-Modal Tree-Structure Embedded Multi-Task Learning , 2017, IEEE Transactions on Intelligent Transportation Systems.

[35]  Florian Franzen,et al.  Traffic Sign Recognition with Neural Networks in the Frequency Domain , 2020, Journal of Physics: Conference Series.

[36]  Anjan Gudigar,et al.  Local texture patterns for traffic sign recognition using higher order spectra , 2017, Pattern Recognit. Lett..

[37]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[38]  Yi Yang,et al.  Towards Real-Time Traffic Sign Detection and Classification , 2016, IEEE Transactions on Intelligent Transportation Systems.

[39]  Klaus Zimmermann,et al.  Towards reliable traffic sign recognition , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[40]  Amara Dinesh Kumar,et al.  Novel Deep Learning Model for Traffic Sign Detection Using Capsule Networks , 2018, ArXiv.

[41]  Zhaohui Wu,et al.  Weakly Supervised Metric Learning for Traffic Sign Recognition in a LIDAR-Equipped Vehicle , 2016, IEEE Transactions on Intelligent Transportation Systems.

[42]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[43]  Andrew McCallum,et al.  Energy and Policy Considerations for Deep Learning in NLP , 2019, ACL.

[44]  Changshui Zhang,et al.  Traffic Sign Recognition With Hinge Loss Trained Convolutional Neural Networks , 2014, IEEE Transactions on Intelligent Transportation Systems.

[45]  N. Nakicenovic,et al.  Six Transformations to achieve the Sustainable Development Goals , 2019, Nature Sustainability.

[46]  Yue Zhou,et al.  Design of Lightweight Convolutional Neural Network Based on Dimensionality Reduction Module , 2019 .

[47]  Shangyou Zeng,et al.  Traffic Sign Recognition with a small convolutional neural network , 2019 .

[48]  Michael Felsberg,et al.  Using Fourier Descriptors and Spatial Models for Traffic Sign Recognition , 2011, SCIA.

[49]  Peter H. N. de With,et al.  Color exploitation in hog-based traffic sign detection , 2010, 2010 IEEE International Conference on Image Processing.

[50]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[51]  Ke Lu,et al.  Sparse-Representation-Based Graph Embedding for Traffic Sign Recognition , 2012, IEEE Transactions on Intelligent Transportation Systems.

[52]  Peter J. Braspenning,et al.  Artificial Neural Networks: An Introduction to ANN Theory and Practice , 1995, Artificial Neural Networks.

[53]  Guanhua Hou,et al.  Measuring and examining traffic sign comprehension with event-related potentials , 2020, Cognition, Technology & Work.

[54]  A. Herbin,et al.  Robust on-vehicle real-time visual detection of American and European speed limit signs, with a modular Traffic Signs Recognition system , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[55]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[56]  Liu Tiantian,et al.  A detection and recognition method for prohibition traffic signs , 2010, 2010 International Conference on Image Analysis and Signal Processing.

[57]  Zengfu Wang,et al.  Real-Time Traffic Sign Recognition Based on Efficient CNNs in the Wild , 2019, IEEE Transactions on Intelligent Transportation Systems.

[58]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  Luc Van Gool,et al.  Traffic sign recognition — How far are we from the solution? , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[60]  Wenyu Liu,et al.  Traffic sign detection and recognition using fully convolutional network guided proposals , 2016, Neurocomputing.

[61]  Luis Moreno,et al.  Road traffic sign detection and classification , 1997, IEEE Trans. Ind. Electron..

[62]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[63]  Jürgen Schmidhuber,et al.  Multi-column deep neural network for traffic sign classification , 2012, Neural Networks.