Semantic segmentation-based traffic sign detection and recognition using deep learning techniques

We present a method for detecting and classifying traffic signs based on two deep neural network architectures. A Fully Convolutional Network (FCN) - based semantic segmentation model is modified to extract traffic sign regions of interest. These regions are further passed to a Convolutional Neural Network (CNN) for traffic sign classification. We propose a novel CNN architecture for the classification step. In evaluating our approach, we contrast the efficiency and the robustness of the deep learning image segmentation approach with classical image processing filters traditionally applied for traffic sign detection. We also show the effectiveness of our CNN-based recognition method by integrating it in our system.

[1]  Luis Miguel Bergasa,et al.  Efficient ConvNet for real-time semantic segmentation , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[2]  Mozhdeh Shahbazi,et al.  Object-based Traffic Sign Detection and Recognition from ‎Integrated Range and Intensity Images , 2011 .

[3]  Atsuto Maki,et al.  A systematic study of the class imbalance problem in convolutional neural networks , 2017, Neural Networks.

[4]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Rishi Kumar,et al.  Hierarchical CNN for traffic sign recognition , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[6]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[7]  Eugenio Culurciello,et al.  ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation , 2016, ArXiv.

[8]  Jitendra Malik,et al.  Learning a classification model for segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[9]  Johannes Stallkamp,et al.  Detection of traffic signs in real-world images: The German traffic sign detection benchmark , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[10]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[11]  Peter Kontschieder,et al.  The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[12]  Johannes Stallkamp,et al.  Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition , 2012, Neural Networks.

[13]  Baoli Li,et al.  Traffic-Sign Detection and Classification in the Wild , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Ting Yu,et al.  Simultaneous image segmentation and 3D plane fitting for RGB-D sensors — An iterative framework , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[15]  Sergey Ioffe,et al.  Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models , 2017, NIPS.

[16]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[17]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[18]  Yann LeCun,et al.  Traffic sign recognition with multi-scale Convolutional Networks , 2011, The 2011 International Joint Conference on Neural Networks.

[19]  Licheng Jiao,et al.  The image segmentation based on optimized spatial feature of superpixel , 2015, J. Vis. Commun. Image Represent..

[20]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Long Liu,et al.  Traffic Signs Detection based on Haar-Like Features and Adaboost Classifier , 2013 .

[22]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Nirvair Neeru,et al.  A Brief Review: Super-Pixel Based Image Segmentation Methods , 2016 .

[24]  Jordi Vitrià,et al.  Traffic Sign Recognition Using Evolutionary Adaboost Detection and Forest-ECOC Classification , 2009, IEEE Transactions on Intelligent Transportation Systems.

[25]  Feng Wu,et al.  Traffic sign recognition using HOG-SVM and grid search , 2014, 2014 12th International Conference on Signal Processing (ICSP).

[26]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..