A Q-learning Foresighted Approach to Ego-efficient Lane Changes of Connected and Automated Vehicles on Freeways

Lane changes are a vital part of vehicle motions on roads, affecting surrounding vehicles locally and traffic flow collectively. In the context of connected and automated vehicles (CAVs), this paper is concerned with the impacts of smart lane changes of CAVs on their own travel performance as well as on the entire traffic flow with the increase of the market penetration rate (MPR). On the basis of intensive microscopic traffic simulation and reinforcement learning technique, an ego-efficient lane-changing strategy was first developed in this work to enable foresighted lane changing decisions for CAVs to improve their travel efficiency. The overall impacts of such smart lane changes on traffic flow of both CAVs and human-driven vehicles were then examined on the same simulation platform, which reflects a real freeway infrastructure with real demands. It was found that smart lane changes were beneficial for both CAVs and the entire traffic flow, if MPR was not more than 60%.

[1]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[2]  Wenchao Xu,et al.  Drone Assisted Vehicular Networks: Architecture, Challenges and Opportunities , 2018, IEEE Network.

[3]  Mashrur Chowdhury,et al.  A Review of Communication, Driver Characteristics, and Controls Aspects of Cooperative Adaptive Cruise Control (CACC) , 2016, IEEE Transactions on Intelligent Transportation Systems.

[4]  Fatima de L. P. Duarte-Figueiredo,et al.  A 5G V2X Ecosystem Providing Internet of Vehicles † , 2019, Sensors.

[5]  Hao Liu,et al.  Modeling impacts of Cooperative Adaptive Cruise Control on mixed traffic flow in multi-lane freeway facilities , 2018, Transportation Research Part C: Emerging Technologies.

[6]  Markos Papageorgiou,et al.  Exploitation of ACC systems towards improved traffic flow efficiency on motorways , 2017, 2017 5th IEEE International Conference on Models and Technologies for Intelligent Transportation Systems (MT-ITS).

[7]  Zuduo Zheng,et al.  Recent developments and research needs in modeling lane changing , 2014 .

[8]  Jacob Holden,et al.  Potentials for Platooning in U.S. Highway Freight Transport , 2017 .

[9]  Vicente Milanés Montero,et al.  Cooperative Adaptive Cruise Control in Real Traffic Situations , 2014, IEEE Transactions on Intelligent Transportation Systems.

[10]  Azzedine Boukerche,et al.  A vehicular network based intelligent lane change assistance protocol for highways , 2017, 2017 IEEE International Conference on Communications (ICC).

[11]  Helbing,et al.  Congested traffic states in empirical observations and microscopic simulations , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[12]  Seiichi Mita,et al.  General behavior and motion model for automated lane change , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[13]  Hui Deng,et al.  Platoon management with cooperative adaptive cruise control enabled by VANET , 2015, Veh. Commun..

[14]  Mikhail Gordon,et al.  Lane Change and Merge Maneuvers for Connected and Automated Vehicles: A Survey , 2016, IEEE Transactions on Intelligent Vehicles.

[15]  Lie Guo,et al.  Trajectory planning and tracking control for autonomous lane change maneuver based on the cooperative vehicle infrastructure system , 2015, Expert Syst. Appl..

[16]  Yizhou Wang,et al.  Autonomous lane-change controller via mixed logical dynamical , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[17]  Markos Papageorgiou,et al.  Overview and analysis of Vehicle Automation and Communication Systems from a motorway traffic management perspective , 2015 .

[18]  Charles Desjardins,et al.  Cooperative Adaptive Cruise Control: A Reinforcement Learning Approach , 2011, IEEE Transactions on Intelligent Transportation Systems.

[19]  Jonas Sjöberg,et al.  Strategic decision making for automated driving on two-lane, one way roads using model predictive control , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[20]  Gábor Orosz,et al.  Dynamics of connected vehicle systems with delayed acceleration feedback , 2014 .

[21]  Meng Wang,et al.  Unravelling effects of cooperative adaptive cruise control deactivation on traffic flow characteristics at merging bottlenecks , 2018, Transportation Research Part C: Emerging Technologies.