Assessment of Adaptive Traffic Signal Control Using Hardware in the Loop Simulation

Adaptive Traffic Signal Control (ATSC) can potentially mitigate traffic congestion. Research and development in the area of ATSC have produced a number of new systems with promising potential, however the performance of these systems under real-life conditions has been always a concern for practitioners as well as researchers, particularly if and how the new systems would be implementable in the field on controllers with specific capabilities and limitations. Therefore, testing and refining new ATSC systems on actual hardware and under representative traffic conditions prior to field implementation is essential to bridge this gap. In this paper, a hardware in-the-loop simulation (HILS) framework is developed to evaluate MARLIN, as an example of a new self-learning ATSC system. HILS is used for evaluating hardware components running the ATSC software in a simulation environment in which an actual traffic signal controller and an embedded computer are physically connected to a microscopic traffic simulator. Our focus is on the development, implementation of the HILS framework and the evaluation of MARLIN, on an intersection that suffers significant traffic fluctuation and delays - at the City of Burlington, Ontario, Canada. The performance of MARLIN-ATSC is demonstrated with HILS, which consists of a PEEK ATC-1000 traffic controller, an embedded computer running the ATSC system, and Paramics microscopic simulation model. HILS results indicated that MARLIN-ATSC has the potential to reduce the intersection average delay by up to 20% on average compared to the optimized and coordinated actuated signal timing plans.

[1]  T. Urbanik,et al.  Reinforcement learning-based multi-agent system for network traffic signal control , 2010 .

[2]  Peter T. Martin,et al.  SCOOT ADAPTIVE SIGNAL CONTROL : AN EVALUATION OF ITS EFFECTIVENESS OVER A RANGE OF CONGESTION INTENSITIES , 2003 .

[3]  Ilsoo Yun,et al.  Evaluation of Adaptive Maximum Feature in Actuated Traffic Controller , 2007 .

[4]  Lu Shoufeng,et al.  Q-Learning for Adaptive Traffic Signal Control Based on Delay Minimization Strategy , 2008, 2008 IEEE International Conference on Networking, Sensing and Control.

[5]  Baher Abdulhai,et al.  Multiagent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers (MARLIN-ATSC): Methodology and Large-Scale Application on Downtown Toronto , 2013, IEEE Transactions on Intelligent Transportation Systems.

[6]  Thomas L. Thorpe.  Vehicle Traffic Light Control Using SARSA , 1997 .

[7]  Kevin N. Balke,et al.  RECENT APPLICATIONS OF HARDWARE-IN-THE-LOOP TRAFFIC SIMULATION , 2000 .

[8]  Tao Li,et al.  Adaptive Dynamic Programming for Multi-intersections Traffic Signal Intelligent Control , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[9]  Eduardo Camponogara,et al.  Distributed Learning Agents in Urban Traffic Control , 2003, EPIA.

[10]  Rahim F Benekohal,et al.  Q-learning and Approximate Dynamic Programming for Traffic Control: A Case Study for an Oversaturated Network , 2012 .

[11]  Vinny Cahill,et al.  A Collaborative Reinforcement Learning Approach to Urban Traffic Control Optimization , 2008, 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[12]  Tien Fang Fwa,et al.  Applications of Advanced Technologies in Transportation Engineering , 2004 .

[13]  P B Mirchandani,et al.  RHODES-ITMS TEMPE FIELD TEST PROJECT: IMPLEMENTATION AND FIELD TESTING OF RHODES, A REAL-TIME TRAFFIC ADAPTIVE CONTROL SYSTEM , 2001 .

[14]  Baher Abdulhai,et al.  Comprehensive Analysis of Reinforcement Learning Methods and Parameters for Adaptive Traffic Signal Control , 2011 .

[15]  Marco Wiering,et al.  Multi-Agent Reinforcement Learning for Traffic Light control , 2000 .

[16]  Shimon Whiteson,et al.  Multiagent Reinforcement Learning for Urban Traffic Control Using Coordination Graphs , 2008, ECML/PKDD.

[17]  Ling Li,et al.  APPLICATION OF THE HARDWARE-IN-THE-LOOP SIMULATION FOR UPDATING THE TRAFFIC SIGNAL TIMING PLANS , 2004 .

[18]  Peter T. Martin,et al.  SCOOT Real-Time Adaptive Control in a CORSIM Simulation Environment , 2000 .

[19]  Susan R Dickey.  Development of Hardware in the Loop Simulation and Paramics/VS-PLUS Integration , 2008 .

[20]  Baher Abdulhai,et al.  Multi-Agent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers (MARLIN-ATSC) , 2012, 2012 15th International IEEE Conference on Intelligent Transportation Systems.

[21]  Michael Hunter,et al.  Hardware-in-the-Loop Simulation Evaluation of Adaptive Signal Control , 2010 .

[22]  Robert L. Bertini,et al.  Using Hardware-in-the-Loop Simulation to Evaluate Signal Control Strategies for Transit Signal Priority , 2005 .

[23]  Ping Yi.  Traffic-control systems , 2014 .

[24]  Jin Yu,et al.  Natural Actor-Critic for Road Traffic Optimisation , 2006, NIPS.

[25]  Baher Abdulhai,et al.  Reinforcement learning for true adaptive traffic signal control , 2003 .