Infinitesimal Perturbation Analysis of Stochastic Flow Models with Delays: Application to Multi-Intersection Traffic Light Control

We extend Stochastic Flow Models (SFMs), which are used for a large class of hybrid systems, by including the delays which typically arise in flow movement. We apply this framework to the multi-intersection traffic light control problem by including transit delays for vehicles moving from one intersection to the next. Using Infinitesimal Perturbation Analysis (IPA) for this SFM with delays, we derive new online gradient estimates of several congestion cost metrics with respect to the controllable green and red cycle lengths. The IPA estimators are used to iteratively adjust light cycle lengths to improve performance and, in conjunction with a standard gradient-based algorithm, to obtain optimal values which adapt to changing traffic conditions. We introduce two new cost metrics to better capture congestion and show that the inclusion of delays in our analysis leads to improved performance relative to models that ignore delays.