Optimizing Traffic Lights with Multi-agent Deep Reinforcement Learning and V2X communication

We consider a system to optimize duration of traffic signals using multi-agent deep reinforcement learning and Vehicle-to-Everything (V2X) communication. This system aims at analyzing independent and shared rewards for multi-agents to control duration of traffic lights. A learning agent traffic light gets information along its lanes within a circular V2X coverage. The duration cycles of traffic light are modeled as Markov decision Processes. We investigate four variations of reward functions. The first two are unshared-rewards: based on waiting number, and waiting time of vehicles between two cycles of traffic light. The third and fourth functions are: shared-rewards based on waiting cars, and waiting time for all agents. Each agent has a memory for optimization through target network and prioritized experience replay. We evaluate multi-agents through the Simulation of Urban MObility (SUMO) simulator. The results prove effectiveness of the proposed system to optimize traffic signals and reduce average waiting cars to 41.5 % as compared to the traditional periodic traffic control system.

[1]  Kuei-Hsiang Chao,et al.  An Intelligent Traffic Flow Control System Based on Radio Frequency Identification and Wireless Sensor Networks , 2014, Int. J. Distributed Sens. Networks.

[2]  Minoru Ito,et al.  Adaptive Traffic Signal Control: Deep Reinforcement Learning Algorithm with Experience Replay and Target Network , 2017, ArXiv.

[3]  F. Webster.  TRAFFIC SIGNAL SETTINGS , 1958 .

[4]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[5]  Baher Abdulhai,et al.  Design of Reinforcement Learning Parameters for Seamless Application of Adaptive Traffic Signal Control , 2014, J. Intell. Transp. Syst..

[6]  Alan J. Miller.  Settings for Fixed-Cycle Traffic Signals , 1963 .

[7]  Saiedeh N. Razavi,et al.  Using a Deep Reinforcement Learning Agent for Traffic Signal Control , 2016, ArXiv.

[8]  Zhu Han,et al.  A Deep Reinforcement Learning Network for Traffic Light Cycle Control , 2018, IEEE Transactions on Vehicular Technology.

[9]  Baher Abdulhai,et al.  Reinforcement learning for true adaptive traffic signal control , 2003 .

[10]  He Jiang,et al.  Neural-Network-Based Robust Control Schemes for Nonlinear Multiplayer Systems With Uncertainties via Adaptive Dynamic Programming , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[11]  John B. Kenney,et al.  Dedicated Short-Range Communications (DSRC) Standards in the United States , 2011, Proceedings of the IEEE.

[12]  Tan Yan,et al.  A Distributed Intersection Management Protocol for Safety, Efficiency, and Driver’s Comfort , 2018, IEEE Internet of Things Journal.

[13]  Li Li,et al.  Traffic signal timing via deep reinforcement learning , 2016, IEEE/CAA Journal of Automatica Sinica.

[14]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[15]  S. Chand,et al.  Adaptive traffic signal control using fuzzy logic , 1993, [Proceedings 1993] Second IEEE International Conference on Fuzzy Systems.

[16]  Marco Wiering,et al.  Multi-Agent Reinforcement Learning for Traffic Light control , 2000 .

[17]  Monireh Abdoos,et al.  Holonic multi-agent system for traffic signals control , 2013, Eng. Appl. Artif. Intell..

[18]  Frans A. Oliehoek,et al.  Video Demo: Deep Reinforcement Learning for Coordination in Traffic Light Control , 2016 .

[19]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[20]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.