Knowledge-based Recurrent Attentive Neural Network for Traffic Sign Detection

Accurate Traffic Sign Detection (TSD) can help drivers make better decision according to the traffic regulations. TSD, regarded as a typical small object detection problem in some way, is fundamental in the field of self-driving and advanced driver assistance systems. However, small object detection is still an open question. In this paper, we proposed a human brain inspired network to handle this problem. Attention mechanism is an essential function of our brain, we used a novel recurrent attentive neural network to improve the detection accuracy in a fine-grained manner. Further, as we human can combine domain specific knowledge and intuitive knowledge to solve tricky tasks, we proposed an assumption that the location of the traffic signs obeys the reverse gaussian distribution, which means the location is around the central bias of every picture. Experimental result shows that our methods achieved better performance than several popular methods used in object detection.

[1]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[2]  Forrest N. Iandola,et al.  SqueezeDet: Unified, Small, Low Power Fully Convolutional Neural Networks for Real-Time Object Detection for Autonomous Driving , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  Rogério Schmidt Feris,et al.  A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection , 2016, ECCV.

[4]  Lars Petersson,et al.  Large scale sign detection using HOG feature variants , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[5]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6]  José Manuel Pastor,et al.  Visual sign information extraction and identification by deformable models for intelligent vehicles , 2004, IEEE Transactions on Intelligent Transportation Systems.

[7]  Sancho Salcedo-Sanz,et al.  A decision support system for the automatic management of keep-clear signs based on support vector machines and geographic information systems , 2010, Expert Syst. Appl..

[8]  Majid Mirmehdi,et al.  Recognizing Text-Based Traffic Signs , 2015, IEEE Transactions on Intelligent Transportation Systems.

[9]  Silvio Savarese,et al.  Subcategory-Aware Convolutional Neural Networks for Object Proposals and Detection , 2016, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[10]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Thomas B. Moeslund,et al.  Vision-Based Traffic Sign Detection and Analysis for Intelligent Driver Assistance Systems: Perspectives and Survey , 2012, IEEE Transactions on Intelligent Transportation Systems.

[12]  Cuneyt Akinlar,et al.  Circular traffic sign recognition empowered by circle detection algorithm , 2013, 2013 21st Signal Processing and Communications Applications Conference (SIU).

[13]  Quoc V. Le,et al.  Listen, attend and spell: A neural network for large vocabulary conversational speech recognition , 2015, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[15]  Trevor Darrell,et al.  Long-term recurrent convolutional networks for visual recognition and description , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[17]  K M Sumi,et al.  Detection and Recognition of Road Traffic Signs - A Survey , 2017 .

[18]  Klaus Zimmermann,et al.  Towards reliable traffic sign recognition , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[19]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[20]  Xiaohui Liu,et al.  Detection, Tracking and Recognition of Traffic Signs from Video Input , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[21]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[22]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[23]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Stefano Ermon,et al.  Label-Free Supervision of Neural Networks with Physics and Domain Knowledge , 2016, AAAI.

[25]  Peng Wang,et al.  Ask Me Anything: Free-Form Visual Question Answering Based on Knowledge from External Sources , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Nilesh J. Uke.  DETECTION, CLASSIFICATION AND RECOGNITION OF ROAD TRAFFIC SIGNS USING COLOR AND SHAPE , 2012 .

[27]  Wei Liu,et al.  DSSD : Deconvolutional Single Shot Detector , 2017, ArXiv.

[28]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[29]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Gang Wang,et al.  Recurrent Attentional Networks for Saliency Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Fatin Zaklouta,et al.  Real-Time Traffic-Sign Recognition Using Tree Classifiers , 2012, IEEE Transactions on Intelligent Transportation Systems.

[32]  Francisco López-Ferreras,et al.  Traffic sign shape classification and localization based on the normalized FFT of the signature of blobs and 2D homographies , 2008, Signal Processing.

[33]  Jean-Luc Starck,et al.  A combined approach for object detection and deconvolution , 2000, Astronomy and Astrophysics Supplement Series.

[34]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[35]  Rita Cucchiara,et al.  Predicting Human Eye Fixations via an LSTM-Based Saliency Attentive Model , 2016, IEEE Transactions on Image Processing.

[36]  Paulo Lobato Correia,et al.  Traffic Sign Recognition Based on Pictogram Contours , 2008, 2008 Ninth International Workshop on Image Analysis for Multimedia Interactive Services.

[37]  Giovanni Pilato,et al.  Road signs recognition using a dynamic pixel aggregation technique in the HSV color space , 2001, Proceedings 11th International Conference on Image Analysis and Processing.

[38]  Lutz Priese,et al.  On hierarchical color segmentation and applications , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[39]  A. Mtibaa,et al.  Automatic detection and recognition of road sign for driver assistance system , 2012, 2012 16th IEEE Mediterranean Electrotechnical Conference.

[40]  Luc Van Gool,et al.  Multi-view traffic sign detection, recognition, and 3D localisation , 2014, 2009 Workshop on Applications of Computer Vision (WACV).

[41]  Visvanathan Ramesh,et al.  A system for traffic sign detection, tracking, and recognition using color, shape, and motion information , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[42]  Dariu Gavrila,et al.  Traffic Sign Recognition Revisited , 1999, DAGM-Symposium.

[43]  Francisco López-Ferreras,et al.  Road-Sign Detection and Recognition Based on Support Vector Machines , 2007, IEEE Transactions on Intelligent Transportation Systems.

[44]  Xiaohong W. Gao,et al.  Recognition of traffic signs based on their colour and shape features extracted using human vision models , 2006, J. Vis. Commun. Image Represent..

[45]  Johannes Stallkamp,et al.  The German Traffic Sign Recognition Benchmark: A multi-class classification competition , 2011, The 2011 International Joint Conference on Neural Networks.