Boosted Genetic Algorithm Using Machine Learning for Traffic Control Optimization

Traffic control optimization is a challenging task for various traffic centers around the world and the majority of existing approaches focus only on developing adaptive methods under normal (recurrent) traffic conditions. Optimizing the control plans when severe incidents occur still remains an open problem, especially when a high number of lanes or entire intersections are affected. This paper aims at tackling this problem and presents a novel methodology for optimizing the traffic signal timings in signalized urban intersections, under non-recurrent traffic incidents. With the purpose of producing fast and reliable decisions, we combine the fast running Machine Learning (ML) algorithms and the reliable Genetic Algorithms (GA) into a single optimization framework. As a benchmark, we first start with deploying a typical GA algorithm by considering the phase duration as the decision variable and the objective function to minimize the total travel time in the network. We fine tune the GA for crossover, mutation, fitness calculation and obtain the optimal parameters. Secondly, we train various machine learning regression models to predict the total travel time of the studied traffic network, and select the best performing regressor which we further hyper-tune to find the optimal training parameters. Lastly, we propose a new algorithm BGA-ML combining the GA algorithm and the extreme-gradient decision-tree, which is the best performing regressor, together in a single optimization framework. Comparison and results show that the new BGA-ML is much faster than the original GA algorithm and can be successfully applied under non-recurrent incident conditions.

[1]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[2]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[3]  J. Brian Gray,et al.  Introduction to Linear Regression Analysis , 2002, Technometrics.

[4]  Marian-Andrei Rizoiu,et al.  Arterial incident duration prediction using a bi-level framework of extreme gradient-tree boosting , 2019, ArXiv.

[5]  A Schadschneider,et al.  Optimizing traffic lights in a cellular automaton model for city traffic. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[6]  Baher Abdulhai,et al.  Multiagent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers (MARLIN-ATSC): Methodology and Large-Scale Application on Downtown Toronto , 2013, IEEE Transactions on Intelligent Transportation Systems.

[7]  Peter Corcoran,et al.  Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning , 2017, ArXiv.

[8]  Hai Yang,et al.  Traffic assignment and signal control in saturated road networks , 1995 .

[9]  J. Friedman.  Stochastic gradient boosting , 2002 .

[10]  Baher Abdulhai,et al.  Design of Reinforcement Learning Parameters for Seamless Application of Adaptive Traffic Signal Control , 2014, J. Intell. Transp. Syst..

[11]  Jinde Cao,et al.  A model and genetic algorithm for area-wide intersection signal optimization under user equilibrium traffic , 2017, Math. Comput. Simul..

[12]  Thomas L. Thorpe,et al.  Traac Light Control Using Sarsa with Three State Representations , 1996 .

[13]  Dipti Srinivasan,et al.  Urban traffic signal control using reinforcement learning agents , 2010 .

[14]  Pravin Varaiya.  Finding and Analyzing True Effect of Non-Recurrent Congestion on Mobilityand Safety , 2007 .

[15]  Wang,et al.  Review of road traffic control strategies , 2003, Proceedings of the IEEE.

[16]  Noe Casas,et al.  Deep Deterministic Policy Gradient for Urban Traffic Light Control , 2017, ArXiv.

[17]  Malik Beshir Malik,et al.  Applied Linear Regression , 2005, Technometrics.

[18]  Kenneth Tze Kin Teo,et al.  Q-Learning Based Traffic Optimization in Management of Signal Timing Plan , 2020 .

[19]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[20]  Michael G.H. Bell,et al.  Traffic signal timing optimisation based on genetic algorithm approach, including drivers’ routing , 2004 .

[21]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[22]  T. Urbanik,et al.  Reinforcement learning-based multi-agent system for network traffic signal control , 2010 .

[23]  M. J. Smith,et al.  Traffic Equilibrium with Responsive Traffic Control , 1993, Transp. Sci..

[24]  Andy Liaw,et al.  Classification and Regression by randomForest , 2007 .

[25]  Baher Abdulhai,et al.  Reinforcement learning for true adaptive traffic signal control , 2003 .

[26]  Stephen G. Ritchie,et al.  A KNOWLEDGE-BASED DECISION SUPPORT ARCHITECTURE FOR ADVANCED TRAFFIC MANAGEMENT , 1990 .

[27]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[28]  Jim Duggan,et al.  An Experimental Review of Reinforcement Learning Algorithms for Adaptive Traffic Signal Control , 2016, Autonomic Road Transport Support Systems.

[29]  David E. Goldberg,et al.  Genetic algorithms and Machine Learning , 1988, Machine Learning.

[30]  Markos Papageorgiou,et al.  Extensions and New Applications of the Traffic-Responsive Urban Control Strategy: Coordinated Signal Control for Urban Networks , 2003 .

[31]  L. Breiman.  Arcing the edge , 1997 .

[32]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[33]  Robert E. Schapire,et al.  A Brief Introduction to Boosting , 1999, IJCAI.

[34]  Chetan Gupta,et al.  Forecasting spatiotemporal impact of traffic incidents for next-generation navigation systems , 2014, Knowledge and Information Systems.

[35]  Alexander Skabardonis,et al.  Measuring Recurrent and Nonrecurrent Traffic Congestion , 2008 .

[36]  T. Bosse,et al.  Video Demo: Deep Reinforcement Learning for Coordination in Traffic Light Control , 2017 .

[37]  Yoav Freund,et al.  A Short Introduction to Boosting , 1999 .

[38]  Marco Wiering,et al.  Multi-Agent Reinforcement Learning for Traffic Light control , 2000 .

[39]  David E. Goldberg,et al.  SIGNAL TIMING DETERMINATION USING GENETIC ALGORITHMS , 1992 .

[40]  Monireh Abdoos,et al.  Holonic multi-agent system for traffic signals control , 2013, Eng. Appl. Artif. Intell..