Q-learning based intelligent multi-objective particle swarm optimization of light control for traffic urban congestion management

Traffic signal operations play an important role in the effective functioning of the urban area. However, due to the increasing number of vehicles and the high dynamic of the traffic network, conventional traffic signal timing methods does not result in an efficient control. One alternative is to let traffic signal controllers learn how to adjust the lights based on the traffic situation. In this paper, we propose a novel multi-objective traffic light control system that is based on an Intelligent Multi-Objective Particle Swarm Optimization (MOPSO) method. We take the average junction waiting time and the flow rate of vehicles on the congested road as two objectives. In the proposed method, we granted the ability of selecting appropriate MOPSO parameters to each agent of the swarm via a novel multi-objective Q-Learning approach. The simulation results demonstrate the efficiency of the proposed system.

[1]  Jonathan E. Fieldsend,et al.  A Multi-Objective Algorithm based upon Particle Swarm Optimisation, an Efficient Data Structure and , 2002 .

[2]  Carlos A. Coello Coello,et al.  Multi-Objective Particle Swarm Optimizers: An Experimental Comparison , 2009, EMO.

[3]  K. Zine-dine,et al.  USING PSO ALGORITHM FOR THE TRAFFIC LIGHTS SETTING PROBLEM WITH CELLULAR AUTOMATON MODEL 1 , 2013 .

[4]  Kenneth Tze Kin Teo,et al.  Q-Learning Traffic Signal Optimization within Multiple Intersections Traffic Network , 2012, 2012 Sixth UKSim/AMSS European Symposium on Computer Modeling and Simulation.

[5]  Ming Li Song,et al.  A Study of Single-Objective Particle Swarm Optimization and Multi-Objective Particle Swarm Optimization , 2014 .

[6]  Lucia Lo Bello,et al.  A novel approach for dynamic traffic lights management based on Wireless Sensor Networks and multiple fuzzy logic controllers , 2015, Expert Syst. Appl..

[7]  Mohammed Alhanjouri,et al.  MULTI -RESOLUTION ANALYSIS FOR MEDICAL IMAGE COMPRESSION , 2011 .

[8]  José García-Nieto,et al.  Swarm intelligence for traffic light scheduling: Application to real urban areas , 2012, Eng. Appl. Artif. Intell..

[9]  Abbas Khosravi,et al.  Q-learning method for controlling traffic signal phase time in a single intersection , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[10]  Jürgen Teich,et al.  Strategies for finding good local guides in multi-objective particle swarm optimization (MOPSO) , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[11]  M Reyes Sierra,et al.  Multi-Objective Particle Swarm Optimizers: A Survey of the State-of-the-Art , 2006 .

[12]  Vinny Cahill,et al.  A Collaborative Reinforcement Learning Approach to Urban Traffic Control Optimization , 2008, 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[13]  Michael N. Vrahatis,et al.  Multi-Objective Particles Swarm Optimization Approaches , 2008 .

[14]  Mojtaba Salehi,et al.  TLCSBFL: A Traffic Lights Control System Based on Fuzzy Logic , 2014 .

[15]  Yin Han,et al.  Intersection Signal Control Approach Based on PSO and Simulation , 2008, 2008 Second International Conference on Genetic and Evolutionary Computing.

[16]  Saeid Nahavandi,et al.  Optimal fuzzy traffic signal controller for an isolated intersection , 2014, 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[17]  Xiao-Hua Yu,et al.  Traffic signal optimization using Ant Colony Algorithm , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).