Enhanced multiagent multi-objective reinforcement learning for urban traffic light control

Traffic light control is one of the major problems in urban areas. This is due to the increasing number of vehicles and the high dynamics of the traffic network. Ordinary methods for traffic light control cause high rate of accidents, waste in time, and affect the environment negatively due to the high rates of fuel consumption. In this paper, we develop an enhanced version of our multiagent multi-objective traffic light control system that is based on a Reinforcement Learning (RL) approach. As a testbed framework for our traffic light controller, we use the open source Green Light District (GLD) vehicle traffic simulator. We analyze and fix some implementation problems in GLD that emerged when applying a more realistic continuous time acceleration model. We propose a new cooperation method between the neighboring traffic light agent controllers using specific learning and exploration rates. Our enhanced traffic light controller minimizes the trip time in major arteries and increases safety in residential areas. In addition, our traffic light controller satisfies green waves for platoons traveling in major arteries and considers as well the traffic environmental impact by keeping the vehicles speeds within the desirable thresholds for lowest fuel consumption. In order to evaluate the enhancements and new methods proposed in this paper, we have added new performance indices to GLD.

[1]  Zhang Yi,et al.  Multiobjective Reinforcement Learning for Traffic Signal Control Using Vehicular Ad Hoc Network , 2010, EURASIP J. Adv. Signal Process..

[2]  A. Koopman,et al.  Simulation and optimization of traffic in a city , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[3]  Li Shui-you.  Immunity genetic algorithms based adaptive control method for urban traffic network signal , 2006 .

[4]  C. P. Pappis,et al.  A Fuzzy Logic Controller for a Trafc Junction , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[5]  Walid Gomaa,et al.  Multi-objective traffic light control system based on Bayesian probability interpretation , 2012, 2012 15th International IEEE Conference on Intelligent Transportation Systems.

[6]  Carlos Gershenson,et al.  Modeling self-organizing traffic lights with elementary cellular automata , 2009, ArXiv.

[7]  Carlos Gershenson,et al.  Self-organizing traffic lights: A realistic simulation , 2013 .

[8]  Zhiyong Liu,et al.  A Survey of Intelligence Methods in Urban Traffic Signal Control , 2007 .

[9]  E. Mamdani,et al.  FUZZY LOGIC CONTROLLER FOR A TRAFFIC JUNCTION , 1977 .

[10]  Helbing,et al.  Congested traffic states in empirical observations and microscopic simulations , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[11]  Ma Shou.  Agent-based learning control method for urban traffic signal of single intersection , 2002 .

[12]  Wu Tie-jun.  Real-time rolling horizon optimization of urban traffic control based on ant algorithm , 2004 .

[13]  Baher Abdulhai,et al.  Reinforcement learning for true adaptive traffic signal control , 2003 .

[14]  Kanok Boriboonsomsin,et al.  Traffic Congestion and Greenhouse Gases , 2009 .

[15]  Thomas L. Thorpe,et al.  Traac Light Control Using Sarsa with Three State Representations , 1996 .

[16]  Marco Wiering,et al.  Multi-Agent Reinforcement Learning for Traffic Light control , 2000 .

[17]  Martin Treiber,et al.  How Much Does Traffic Congestion Increase Fuel Consumption and Emissions? Applying Fuel Consumption Model to NGSIM Trajectory Data , 2008 .

[18]  M.A. Khamis,et al.  Adaptive traffic control system based on Bayesian probability interpretation , 2012, 2012 Japan-Egypt Conference on Electronics, Communications and Computers.