Traffic Sign Detection Under Challenging Conditions: A Deeper Look into Performance Variations and Spectral Characteristics

Traffic signs are critical for maintaining the safety and efficiency of our roads. Therefore, we need to carefully assess the capabilities and limitations of automated traffic sign detection systems. Existing traffic sign datasets are limited in terms of type and severity of challenging conditions. Metadata corresponding to these conditions are unavailable and it is not possible to investigate the effect of a single factor because of the simultaneous changes in numerous conditions. To overcome the shortcomings in existing datasets, we introduced the CURE-TSD-Real dataset, which is based on simulated challenging conditions that correspond to adversaries that can occur in real-world environments and systems. We test the performance of two benchmark algorithms and show that severe conditions can result in an average performance degradation of 29% in precision and 68% in recall. We investigate the effect of challenging conditions through spectral analysis and show that the challenging conditions can lead to distinct magnitude spectrum characteristics. Moreover, we show that mean magnitude spectrum of changes in video sequences under challenging conditions can be an indicator of detection performance. The CURE-TSD-Real dataset is available online at https://github.com/olivesgatech/CURE-TSD.

[1]  Gregory K. Wallace,et al.  The JPEG still picture compression standard , 1992 .

[2]  Johannes Stallkamp,et al.  Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition , 2012, Neural Networks.

[3]  Loong Fah Cheong,et al.  Rolling-Shutter-Aware Differential SfM and Image Rectification , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[4]  Baoli Li,et al.  Traffic-Sign Detection and Classification in the Wild , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Nanning Zheng,et al.  Knowledge-based Recurrent Attentive Neural Network for Traffic Sign Detection , 2018, ArXiv.

[6]  Johannes Stallkamp,et al.  Detection of traffic signs in real-world images: The German traffic sign detection benchmark , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[7]  Thomas B. Moeslund,et al.  Vision-Based Traffic Sign Detection and Analysis for Intelligent Driver Assistance Systems: Perspectives and Survey , 2012, IEEE Transactions on Intelligent Transportation Systems.

[8]  Ghassan Al-Regib,et al.  Challenging Environments for Traffic Sign Detection: Reliability Assessment under Inclement Conditions , 2019, ArXiv.

[9]  Jian Sun,et al.  Bundled camera paths for video stabilization , 2013, ACM Trans. Graph..

[10]  Li Chen,et al.  Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compression , 2017, ArXiv.

[11]  Nanning Zheng,et al.  Knowledge-based Recurrent Attentive Neural Network for Small Object Detection , 2018 .

[12]  Ghassan Al-Regib,et al.  Object Recognition Under Multifarious Conditions: A Reliability Analysis and a Feature Similarity-Based Performance Estimation , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[13]  J. H. van Hateren,et al.  Modelling the Power Spectra of Natural Images: Statistics and Information , 1996, Vision Research.

[14]  David A. Forsyth,et al.  NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles , 2017, ArXiv.

[15]  Antonio Torralba,et al.  Statistics of natural image categories , 2003, Network.

[16]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[17]  Yi Yang,et al.  Towards Real-Time Traffic Sign Detection and Classification , 2016, IEEE Transactions on Intelligent Transportation Systems.

[18]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Ghassan Al-Regib,et al.  Traffic Signs in the Wild: Highlights from the IEEE Video and Image Processing Cup 2017 Student Competition [SP Competitions] , 2018, IEEE Signal Process. Mag..

[20]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[21]  Ghassan Al-Regib,et al.  CURE-TSR: Challenging Unreal and Real Environments for Traffic Sign Recognition , 2017, ArXiv.

[22]  Pierre Charbonnier,et al.  Road Sign Detection in Images: A Case Study , 2010, 2010 20th International Conference on Pattern Recognition.

[23]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[24]  Luc Van Gool,et al.  Multi-view traffic sign detection, recognition, and 3D localisation , 2014, Machine Vision and Applications.

[25]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Bin Fan,et al.  Traffic Sign Recognition Using a Multi-Task Convolutional Neural Network , 2018, IEEE Transactions on Intelligent Transportation Systems.

[27]  Jian Sun,et al.  SteadyFlow: Spatially Smooth Optical Flow for Video Stabilization , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Deva Ramanan,et al.  Efficiently Scaling up Crowdsourced Video Annotation , 2012, International Journal of Computer Vision.

[29]  Nicolai Petkov,et al.  Distance sets for shape filters and shape recognition , 2003, IEEE Trans. Image Process..

[30]  Jianming Zhang,et al.  A Real-Time Chinese Traffic Sign Detection Algorithm Based on Modified YOLOv2 , 2017, Algorithms.

[31]  Ghassan Al-Regib,et al.  Distorted Representation Space Characterization Through Backpropagated Gradients , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[32]  Johannes Stallkamp,et al.  The German Traffic Sign Recognition Benchmark: A multi-class classification competition , 2011, The 2011 International Joint Conference on Neural Networks.

[33]  Ghassan Al-Regib,et al.  CURE-OR: Challenging Unreal and Real Environments for Object Recognition , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[34]  Michael Felsberg,et al.  Using Fourier Descriptors and Spatial Models for Traffic Sign Recognition , 2011, SCIA.

[35]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[36]  Ghassan Al-Regib,et al.  Semantically Interpretable and Controllable Filter Sets , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[37]  Yi Yang,et al.  Towards real-time traffic sign detection and classification , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).