Intelligent Traffic Light Control of Isolated Intersections Using Machine Learning Methods

Traffic congestion is one of the major problems in modern cities. This study applies machine learning methods to determine green times in order to minimize in an isolated intersection. Q-learning and neural networks are applied here to set signal light times and minimize total delays. It is assumed that an intersection behaves in a similar fashion to an intelligent agent learning how to set green times in each cycle based on traffic information. Here, a comparison between Q-learning and neural network is presented. In Q-learning, considering continuous green time requires a large state space, making the learning process practically impossible. In contrast to Q-learning methods, the neural network model can easily set the appropriate green time to fit the traffic demand. The performance of the proposed neural network is compared with two traditional alternatives for controlling traffic lights. Simulation results indicate that the application of the proposed method greatly reduces the total delay in the network compared to the alternative methods.

[1]  Mohd Nasir Taib,et al.  Classification of EEG Spectrogram Image with ANN Approach for Brainwave Balancing Application , 2020 .

[2]  Kenneth Tze Kin Teo,et al.  Q-Learning Based Traffic Optimization in Management of Signal Timing Plan , 2020 .

[3]  Paulo Martins Engel,et al.  Improving reinforcement learning with context detection , 2006, AAMAS '06.

[4]  Monireh Abdoos,et al.  Traffic light control in non-stationary environments based on multi agent Q-learning , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[5]  Zhang Yi,et al.  Multiobjective Reinforcement Learning for Traffic Signal Control Using Vehicular Ad Hoc Network , 2010, EURASIP J. Adv. Signal Process..

[6]  T. Urbanik,et al.  Reinforcement learning-based multi-agent system for network traffic signal control , 2010 .

[7]  D. C. Chin,et al.  Traffic-responsive signal timing for system-wide traffic control , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[8]  Dipti Srinivasan,et al.  Multi-Agent System in Urban Traffic Signal Control , 2010, IEEE Computational Intelligence Magazine.

[9]  Lei Yang,et al.  An Approach for Short Term Traffic Flow Forecasting Based on Genetic Neural Network , 2013 .

[10]  Jianqiang Yi,et al.  A comparative study of urban traffic signal control with reinforcement learning and Adaptive Dynamic Programming , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[11]  Ana L. C. Bazzan,et al.  A Distributed Approach for Coordination of Traffic Signal Agents , 2005, Autonomous Agents and Multi-Agent Systems.

[12]  Alexander S. Poznyak,et al.  Urban Traffic Control Problem: a Game Theory Approach , 2008 .

[13]  A. Koopman,et al.  Simulation and optimization of traffic in a city , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[14]  J. Chinrungrueng,et al.  Performance Comparison between Queueing Theoretical Optimality and Q-Learning Approach for Intersection Traffic Signal Control , 2012, 2012 Fourth International Conference on Computational Intelligence, Modelling and Simulation.

[15]  Shalabh Bhatnagar,et al.  Reinforcement Learning With Function Approximation for Traffic Signal Control , 2011, IEEE Transactions on Intelligent Transportation Systems.

[16]  Baher Abdulhai,et al.  Reinforcement learning for true adaptive traffic signal control , 2003 .

[17]  Dipti Srinivasan,et al.  Neural Networks for Real-Time Traffic Signal Control , 2006, IEEE Transactions on Intelligent Transportation Systems.

[18]  James C. Spall,et al.  Evaluation of system-wide traffic signal control using stochastic optimization and neural networks , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[19]  Ella Bingham.  Reinforcement learning in neurofuzzy traffic signal control , 2001, Eur. J. Oper. Res..

[20]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[21]  Qian Xia,et al.  An Optimized Design Scheme for Roundabout Traffic Management , 2012 .

[22]  Yumei Zhang,et al.  A stochastic adaptive control model for isolated intersections , 2007, 2007 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[23]  Emile H. L. Aarts,et al.  Global optimization and simulated annealing , 1991, Math. Program..