Deep Q learning-based traffic signal control algorithms: Model development and evaluation with field data

Abstract To contend traffic congestion on urban networks, existing studies have made great efforts to develop traffic-responsive signal timing algorithms in the last decade. More recently, as an alternative to conventional model-based algorithms, machine learning-based methods have been tested on traffic light timing problems and show promising potentials. However, many researchers and practitioners still questioned the feasibility and applicability of adopting machine learning techniques in the ATSC domain. One of the reasons is that these methods assumed flawless detectors and heavily relied on simulators for training and evaluations. To address such a critical concern, this article customizes a Deep Q-learning Learning (DQL) method to optimize traffic signal timings at urban intersections, where the partial observations from identity-based detectors are inputs, and the green splits are outputs. A simulation-free data-driven prediction model is also developed to train the DQL with reduced computational time. Then the machine learning-based methods are evaluated on a real-world case with Automatic Number-Plate Recognition (ANPR) data. Experiments show the proposed data-driven model can predict the traffic state in limited computational time, and the DQL algorithm is 3.9% better than the field experiment performance from the adaptive control system, SCOOT, and 22% better than the time-of-day plan by SYNCHRO. The results indicate the DQL methods can only yield marginal improvement with restrictive input and output settings in congested traffic flow in comparison to the conventional adaptive method.