IntelliLight: A Reinforcement Learning Approach for Intelligent Traffic Light Control

The intelligent traffic light control is critical for an efficient transportation system. While existing traffic lights are mostly operated by hand-crafted rules, an intelligent traffic light control system should be dynamically adjusted to real-time traffic. There is an emerging trend of using deep reinforcement learning technique for traffic light control and recent studies have shown promising results. However, existing studies have not yet tested the methods on the real-world traffic data and they only focus on studying the rewards without interpreting the policies. In this paper, we propose a more effective deep reinforcement learning model for traffic light control. We test our method on a large-scale real traffic dataset obtained from surveillance cameras. We also show some interesting case studies of policies learned from the real data.

[1]  F. Webster.  TRAFFIC SIGNAL SETTINGS , 1958 .

[2]  Baher Abdulhai,et al.  Multiagent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers (MARLIN-ATSC): Methodology and Large-Scale Application on Downtown Toronto , 2013, IEEE Transactions on Intelligent Transportation Systems.

[3]  J. Caplan,et al.  Building a memory palace in minutes: equivalent memory performance using virtual versus conventional environments with the Method of Loci. , 2012, Acta psychologica.

[4]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[5]  Monireh Abdoos,et al.  Holonic multi-agent system for traffic signals control , 2013, Eng. Appl. Artif. Intell..

[6]  Frans A. Oliehoek,et al.  Video Demo: Deep Reinforcement Learning for Coordination in Traffic Light Control , 2016 .

[7]  Shimon Whiteson,et al.  Traffic Light Control by Multiagent Reinforcement Learning Systems , 2010, Interactive Collaborative Information Systems.

[8]  T. Urbanik,et al.  Reinforcement learning-based multi-agent system for network traffic signal control , 2010 .

[9]  Minoru Ito,et al.  Adaptive Traffic Signal Control: Deep Reinforcement Learning Algorithm with Experience Replay and Target Network , 2017, ArXiv.

[10]  Frans A. Oliehoek,et al.  Coordinated Deep Reinforcement Learners for Traffic Light Control , 2016 .

[11]  D. Panescu,et al.  Emerging Technologies , 2008, IEEE Engineering in Medicine and Biology Magazine.

[12]  Saiedeh N. Razavi,et al.  Using a Deep Reinforcement Learning Agent for Traffic Signal Control , 2016, ArXiv.

[13]  Alan J. Miller.  Settings for Fixed-Cycle Traffic Signals , 1963 .

[14]  Baher Abdulhai,et al.  Reinforcement learning for true adaptive traffic signal control , 2003 .

[15]  Mengqi Liu,et al.  Cooperative Deep Reinforcement Learning for Tra ic Signal Control , 2017 .

[16]  Francois Dion,et al.  COMPARISON OF DELAY ESTIMATES AT UNDER-SATURATED AND OVER-SATURATED PRE-TIMED SIGNALIZED INTERSECTIONS , 2004 .

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[19]  Stéphane Lafortune,et al.  Adaptive Look-ahead Optimization of Traffic Signals , 1999, J. Intell. Transp. Syst..

[20]  Li Li,et al.  Traffic signal timing via deep reinforcement learning , 2016, IEEE/CAA Journal of Automatica Sinica.

[21]  Shimon Whiteson,et al.  Multiagent Reinforcement Learning for Urban Traffic Control Using Coordination Graphs , 2008, ECML/PKDD.

[22]  Jim Duggan,et al.  An Experimental Review of Reinforcement Learning Algorithms for Adaptive Traffic Signal Control , 2016, Autonomic Road Transport Support Systems.

[23]  Marco Wiering,et al.  Multi-Agent Reinforcement Learning for Traffic Light control , 2000 .

[24]  Carlos Gershenson,et al.  Self-organizing traffic lights: A realistic simulation , 2006, Advances in Applied Self-organizing Systems.