Traffic Light Detection Considering Color Saturation Using In-Vehicle Stereo Camera

One of the major causes of traffic accidents according to the statistical report on traffic accidents in Japan is the disregard of traffic lights by drivers. It would be useful if driving support systems could detect and recognize traffic lights and give appropriate information to drivers. Although many studies on intelligent transportation systems have been conducted, the detection of traffic lights using images remains a difficult problem. This is because traffic lights are very small as compared to other objects and there are many objects similar to traffic lights in the road environment. In addition, the pixel colors of traffic lights are easily over-saturated, which renders traffic light detection using color information difficult. The rapid deployment of the new LED traffic lights has led to a new problem. Since LED lights blink at high frequency, if they are captured by a digital video camera, there are frames in which all the traffic lights appear to be turned off. It is impossible to detect traffic lights in these frames by searching the ordinary color of traffic lights. In this paper, we focus on the stable detection of traffic lights, even when they are blinking or when their colors are over-saturated. A method for detecting candidate traffic lights utilizing intensity information together with color information is proposed for handling over-saturated pixels. To exclude candidates that are not traffic lights efficiently, the sizes of the detected candidates are calculated using a stereo image. In addition, we introduce tracking with a Kalman filter to avoid incorrect detection and achieve stable detection of blinking lights. The experimental results using video sequences taken by an in-vehicle stereo camera verify the efficacy of the proposed approaches.