DeepTLR: A single deep convolutional network for detection and classification of traffic lights

Reliable real-time detection of traffic lights is a major concern for the task of autonomous driving. As deep convolutional networks have proven to be a powerful tool in visual object detection, we propose DeepTLR, a camera-based system for real-time detection and classification of traffic lights. Detection and state classification are realized using a single deep convolutional network. DeepTLR does not use any prior knowledge about traffic light locations. Also the detection is executed frame by frame without using temporal information. It is able to detect traffic lights on the whole camera image without any presegmentation. This is achieved by classifying each fine-grained pixel region of the input image and performing a bounding box regression on regions of each class. We show that our algorithm is able to run on frame-rates required for real-time applications while reaching notable results.

[1]  F. Lindner,et al.  Robust recognition of traffic signals , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[2]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[3]  Zheng Liu,et al.  Traffic light recognition in varying illumination using deep learning and saliency map , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[4]  Zheng Liu,et al.  Saliency Map Generation by the Convolutional Neural Network for Real-Time Traffic Light Detection Using Template Matching , 2015, IEEE Transactions on Computational Imaging.

[5]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[6]  Fawzi Nashashibi,et al.  Real time visual traffic lights recognition based on Spot Light Detection and adaptive traffic lights templates , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[7]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Rashid Ansari,et al.  A vision based system for traffic lights recognition , 2015, 2015 SAI Intelligent Systems Conference (IntelliSys).

[9]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[10]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[11]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[12]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[13]  Johann Marius Zöllner,et al.  Visual state estimation of traffic lights using hidden Markov models , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[14]  Fawzi Nashashibi,et al.  Traffic light recognition using image processing compared to learning processes , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Slobodan Ilic,et al.  Semantic Segmentation Based Traffic Light Detection at Day and at Night , 2015, GCPR.

[16]  Fernando A. Mujica,et al.  An Empirical Evaluation of Deep Learning on Highway Driving , 2015, ArXiv.

[17]  Evangelos Dermatas,et al.  Traffic Lights Detection in Adverse Conditions using Color, Symmetry and Spatiotemporal Information , 2012, VISAPP.