Assessment of self-learning adaptive traffic signal control on congested urban areas: independent versus coordinated perspectives

In this paper, we introduce a simulation testbed framework to evaluate the performance of a self-learning adaptive traffic signal control system. The core contribution of this paper is the assessment of the system’s two modes of operations (independent versus coordinated) under different congestion levels and network configurations. The insights and conclusions of the paper are based on the synergetic effect of the following: (1) appropriate design of the adaptive system parameters, (2) seamless design of generic interfaces between the adaptive system and the simulation environment using application programming interfaces, (3) rigorously calibrated simulation model and a comprehensive set of performance and environmental measures, and (4) investigation of the system components required for building a complete functioning system in the field. The system was designed and lab-tested on two case studies in the City of Burlington, Ontario. The intersections were designed and operated using the adaptive system ...

[1]  Nathan H. Gartner,et al.  OPAC: A DEMAND-RESPONSIVE STRATEGY FOR TRAFFIC SIGNAL CONTROL , 1983 .

[2]  R Bissessar.  SCOOT in Toronto - 20 Years Later , 2013 .

[3]  Tao Li,et al.  Adaptive Dynamic Programming for Multi-intersections Traffic Signal Intelligent Control , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[4]  Eduardo Camponogara,et al.  Distributed Learning Agents in Urban Traffic Control , 2003, EPIA.

[5]  Matthew Barth,et al.  Modal Emissions Modeling: A Physical Approach , 1996 .

[6]  Jin Yu,et al.  Natural Actor-Critic for Road Traffic Optimisation , 2006, NIPS.

[7]  Marco Wiering,et al.  Multi-Agent Reinforcement Learning for Traffic Light control , 2000 .

[8]  Baher Abdulhai,et al.  Reinforcement learning for true adaptive traffic signal control , 2003 .

[9]  Rahim F Benekohal,et al.  Q-learning and Approximate Dynamic Programming for Traffic Control: A Case Study for an Oversaturated Network , 2012 .

[10]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[11]  Ana L. C. Bazzan,et al.  Opportunities for multiagent systems and multiagent reinforcement learning in traffic control , 2009, Autonomous Agents and Multi-Agent Systems.

[12]  T. Urbanik,et al.  Reinforcement learning-based multi-agent system for network traffic signal control , 2010 .

[13]  R D Bretherton,et al.  SCOOT-a Traffic Responsive Method of Coordinating Signals , 1981 .

[14]  Pitu B. Mirchandani,et al.  HIERARCHICAL FRAMEWORK FOR REAL-TIME TRAFFIC CONTROL , 1992 .

[15]  Baher Abdulhai,et al.  Reinforcement learning: Introduction to theory and potential for transport applications , 2003 .

[16]  Baher Abdulhai,et al.  Comprehensive Analysis of Reinforcement Learning Methods and Parameters for Adaptive Traffic Signal Control , 2011 .

[17]  Vinny Cahill,et al.  A Collaborative Reinforcement Learning Approach to Urban Traffic Control Optimization , 2008, 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[18]  Mohamed Wahba,et al.  MILATRAS: MIcrosimulation Learning-based Approach to TRansit ASsignment , 2009 .

[19]  Lin Zhang,et al.  Signal Control for Oversaturated Intersections Using Fuzzy Logic , 2008 .

[20]  Shimon Whiteson,et al.  Multiagent Reinforcement Learning for Urban Traffic Control Using Coordination Graphs , 2008, ECML/PKDD.

[21]  Baher Abdulhai,et al.  An agent-based learning towards decentralized and coordinated traffic signal control , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[22]  Lu Shoufeng,et al.  Q-Learning for Adaptive Traffic Signal Control Based on Delay Minimization Strategy , 2008, 2008 IEEE International Conference on Networking, Sensing and Control.

[23]  A.G. Sims,et al.  The Sydney coordinated adaptive traffic (SCAT) system philosophy and benefits , 1980, IEEE Transactions on Vehicular Technology.

[24]  Alexander Skabardonis,et al.  Guidelines for Calibration of Microsimulation Models: Framework and Applications , 2004 .

[25]  Philip J Tarnoff,et al.  DEVELOPMENT OF ADVANCED TRAFFIC SIGNAL CONTROL STRATEGIES FOR INTELLIGENT TRANSPORTATION SYSTEMS: MULTILEVEL DESIGN , 1995 .

[26]  Ennio Cascetta,et al.  Transportation Systems Engineering: Theory and Methods , 2001 .

[27]  Baher Abdulhai,et al.  Multiagent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers (MARLIN-ATSC): Methodology and Large-Scale Application on Downtown Toronto , 2013, IEEE Transactions on Intelligent Transportation Systems.