Rainbow Deep Reinforcement Learning Agent for Improved Solution of the Traffic Congestion

While traffic congestion hits severely the world economy, adaptive traffic signal systems would efficiently provide potential solutions. In this paper, we propose a deep reinforcement learning system to control the signal lights in an isolated intersection. The proposed system uses a deep convolutional neural network to extract the crucial features from the environment state that is described by raw traffic information; i.e., vehicles positions, speeds, and waiting times. Besides, the system utilizes a multi-objective reward and the Rainbow agent which provides further space of enhancements to the conventional Deep Q-Networks agent. Extensive experiments illustrate that our proposed deep framework outperforms the baseline under a number of settings and traffic measures, including trip time, waiting time, fuel consumption, and stability.

[1]  Vinny Cahill,et al.  A Collaborative Reinforcement Learning Approach to Urban Traffic Control Optimization , 2008, 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[2]  Tom Schaul,et al.  Prioritized Experience Replay , 2015, ICLR.

[3]  Marc G. Bellemare,et al.  A Distributional Perspective on Reinforcement Learning , 2017, ICML.

[4]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[5]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[6]  Tom Schaul,et al.  Rainbow: Combining Improvements in Deep Reinforcement Learning , 2017, AAAI.

[7]  R D Bretherton,et al.  SCOOT-a Traffic Responsive Method of Coordinating Signals , 1981 .

[8]  Thomas L. Thorpe,et al.  Traac Light Control Using Sarsa with Three State Representations , 1996 .

[9]  Demis Hassabis,et al.  Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm , 2017, ArXiv.

[10]  Saiedeh N. Razavi,et al.  Using a Deep Reinforcement Learning Agent for Traffic Signal Control , 2016, ArXiv.

[11]  P. Lowrie.  SCATS: Sydney Co-Ordinated Adaptive Traffic System: a traffic responsive method of controlling urban traffic , 1990 .

[12]  Daniel Krajzewicz,et al.  Recent Development and Applications of SUMO - Simulation of Urban MObility , 2012 .

[13]  Lin Zhang,et al.  Signal Control for Oversaturated Intersections Using Fuzzy Logic , 2008 .

[14]  Peter Corcoran,et al.  Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning , 2017, ArXiv.

[15]  Li Li,et al.  Traffic signal timing via deep reinforcement learning , 2016, IEEE/CAA Journal of Automatica Sinica.

[16]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[17]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[18]  Kevin Fehon.  Adaptive Traffic Signals Are we missing the boat , 2004 .

[19]  Frans A. Oliehoek,et al.  Coordinated Deep Reinforcement Learners for Traffic Light Control , 2016 .

[20]  Baher Abdulhai,et al.  Reinforcement learning for true adaptive traffic signal control , 2003 .