Stochastic flow models with delays, blocking and applications to multi-intersection traffic light control

We extend Stochastic Flow Models (SFMs), used for a large class of discrete event and hybrid systems, by including the delays which typically arise in flow movements, as well as blocking effects due to space constraints. We apply this framework to the multi-intersection traffic light control problem by including transit delays for vehicles moving from one intersection to the next and possible blocking between two intersections. Using Infinitesimal Perturbation Analysis (IPA) for this SFM with delays and possible blocking, we derive new on-line gradient estimates of several congestion cost metrics with respect to the controllable green and red cycle lengths. The IPA estimators are used to iteratively adjust light cycle lengths to improve performance and, in conjunction with a standard gradient-based algorithm, to obtain optimal values which adapt to changing traffic conditions. We introduce two new cost metrics to better capture congestions and show that the inclusion of delays and possible blocking in our analysis lead to improved performance relative to models that ignore delays and/or blocking effects.