Vision-based traffic light detection has been widely studied over the past decade. However, it is still a challenging task to build a real-time and robust classifier-based detector without a high dependency on prior knowledge. In this paper, we have a deep look at the design of features and detection mechanism in the domain of traffic light detection; propose a multi-scale and multi-phase detector based on aggregate channel features and boosted trees classifier. Evaluation is done on Daimler, LISA and LaRA datasets, which shows high average-recall and speed. Code has been made publicly available.