A Reinforcement Learning Approach to Smart Lane Changes of Self-driving Cars

Lane changes are a vital part of vehicle motions on roads, affecting surrounding vehicles locally and traffic flow collectively. In the context of connected and automated vehicles (CAVs), this paper is concerned with the impacts of smart lane changes of CAVs on their own travel performance as well as on the entire traffic flow with the increase of the market penetration rate (MPR). On the basis of intensive microscopic traffic simulation and reinforcement learning technique, a selfish lane-changing strategy was first developed in this work to enable foresighted lane changing decisions for CAVs to improve their travel efficiency. The overall impacts of such smart lane changes on traffic flow of both CAVs and human-driven vehicles were then examined on the same simulation platform. It was found that smart lane changes were beneficial for both CAVs and the entire traffic flow, if MPR was not more than 60%.

[1]  Helbing,et al.  Congested traffic states in empirical observations and microscopic simulations , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[2]  Markos Papageorgiou,et al.  Overview and analysis of Vehicle Automation and Communication Systems from a motorway traffic management perspective , 2015 .

[3]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[4]  Gábor Orosz,et al.  Dynamics of connected vehicle systems with delayed acceleration feedback , 2014 .

[5]  Yizhou Wang,et al.  Autonomous lane-change controller via mixed logical dynamical , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[6]  Hao Liu,et al.  Modeling impacts of Cooperative Adaptive Cruise Control on mixed traffic flow in multi-lane freeway facilities , 2018, Transportation Research Part C: Emerging Technologies.

[7]  Lie Guo,et al.  Trajectory planning and tracking control for autonomous lane change maneuver based on the cooperative vehicle infrastructure system , 2015, Expert Syst. Appl..

[8]  Hui Deng,et al.  Platoon management with cooperative adaptive cruise control enabled by VANET , 2015, Veh. Commun..

[9]  Mashrur Chowdhury,et al.  A Review of Communication, Driver Characteristics, and Controls Aspects of Cooperative Adaptive Cruise Control (CACC) , 2016, IEEE Transactions on Intelligent Transportation Systems.

[10]  Markos Papageorgiou,et al.  Exploitation of ACC systems towards improved traffic flow efficiency on motorways , 2017, 2017 5th IEEE International Conference on Models and Technologies for Intelligent Transportation Systems (MT-ITS).

[11]  Zuduo Zheng,et al.  Recent developments and research needs in modeling lane changing , 2014 .

[12]  Mikhail Gordon,et al.  Lane Change and Merge Maneuvers for Connected and Automated Vehicles: A Survey , 2016, IEEE Transactions on Intelligent Vehicles.

[13]  Azzedine Boukerche,et al.  A vehicular network based intelligent lane change assistance protocol for highways , 2017, 2017 IEEE International Conference on Communications (ICC).

[14]  Charles Desjardins,et al.  Cooperative Adaptive Cruise Control: A Reinforcement Learning Approach , 2011, IEEE Transactions on Intelligent Transportation Systems.

[15]  Jacob Holden,et al.  Potentials for Platooning in U.S. Highway Freight Transport , 2017 .

[16]  Vicente Milanés Montero,et al.  Cooperative Adaptive Cruise Control in Real Traffic Situations , 2014, IEEE Transactions on Intelligent Transportation Systems.

[17]  Seiichi Mita,et al.  General behavior and motion model for automated lane change , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[18]  Meng Wang,et al.  Unravelling effects of cooperative adaptive cruise control deactivation on traffic flow characteristics at merging bottlenecks , 2018, Transportation Research Part C: Emerging Technologies.

[19]  Jonas Sjöberg,et al.  Strategic decision making for automated driving on two-lane, one way roads using model predictive control , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).