Decision-Making System for Lane Change Using Deep Reinforcement Learning in Connected and Automated Driving

Lane changing systems have consistently received attention in the fields of vehicular communication and autonomous vehicles. In this paper, we propose a lane change system that combines deep reinforcement learning and vehicular communication. A host vehicle, trying to change lanes, receives the state information of the host vehicle and a remote vehicle that are both equipped with vehicular communication devices. A deep deterministic policy gradient learning algorithm in the host vehicle determines the high-level action of the host vehicle from the state information. The proposed system learns straight-line driving and collision avoidance actions without vehicle dynamics knowledge. Finally, we consider the update period for the state information from the host and remote vehicles.

[1]  Anil A. Bharath,et al.  Deep Reinforcement Learning: A Brief Survey , 2017, IEEE Signal Processing Magazine.

[2]  Ashish Kapoor,et al.  AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles , 2017, FSR.

[3]  Marc Peter Deisenroth,et al.  Deep Reinforcement Learning: A Brief Survey , 2017, IEEE Signal Processing Magazine.

[4]  Hussein Zedan,et al.  A comprehensive survey on vehicular Ad Hoc network , 2014, J. Netw. Comput. Appl..

[5]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[6]  Hieu Nguyen,et al.  A Semi-Empirical Performance Study of Two-Hop DSRC Message Relaying at Road Intersections , 2018, Inf..

[7]  Mikhail Gordon,et al.  Lane Change and Merge Maneuvers for Connected and Automated Vehicles: A Survey , 2016, IEEE Transactions on Intelligent Vehicles.

[8]  Sergey Levine,et al.  Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Long-Ji Lin,et al.  Reinforcement learning for robots using neural networks , 1992 .

[10]  Carl-Johan Hoel,et al.  Automated Speed and Lane Change Decision Making using Deep Reinforcement Learning , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[11]  Jonas Fredriksson,et al.  If, When, and How to Perform Lane Change Maneuvers on Highways , 2016, IEEE Intelligent Transportation Systems Magazine.

[12]  András Kovács,et al.  Enhancements of V2X communication in support of cooperative autonomous driving , 2015, IEEE Communications Magazine.

[13]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[14]  K. Madhava Krishna,et al.  Overtaking Maneuvers in Simulated Highway Driving using Deep Reinforcement Learning , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[15]  Ali Farhadi,et al.  Target-driven visual navigation in indoor scenes using deep reinforcement learning , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Kikuo Fujimura,et al.  Tactical Decision Making for Lane Changing with Deep Reinforcement Learning , 2017 .

[17]  Yong Liang Guan,et al.  Efficient Real-Time Coding-Assisted Heterogeneous Data Access in Vehicular Networks , 2018, IEEE Internet of Things Journal.

[18]  Francesco Borrelli,et al.  Scenario Model Predictive Control for Lane Change Assistance and Autonomous Driving on Highways , 2017, IEEE Intelligent Transportation Systems Magazine.

[19]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[20]  Javier Alonso-Mora,et al.  Planning and Decision-Making for Autonomous Vehicles , 2018, Annu. Rev. Control. Robotics Auton. Syst..

[21]  Matthias Althoff,et al.  High-level Decision Making for Safe and Reasonable Autonomous Lane Changing using Reinforcement Learning , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[22]  Johann Marius Zöllner,et al.  Learning how to drive in a real world simulation with deep Q-Networks , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[23]  István Z. Kovács,et al.  Geolocation-Based Access for Vehicular Communications: Analysis and Optimization via Stochastic Geometry , 2016, IEEE Transactions on Vehicular Technology.

[24]  Ching-Yao Chan,et al.  Formulation of deep reinforcement learning architecture toward autonomous driving for on-ramp merge , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[25]  John E. R. Staddon,et al.  The dynamics of behavior: Review of Sutton and Barto: Reinforcement Learning : An Introduction (2 nd ed.) , 2020 .

[26]  Jie Zhang,et al.  Enhanced Collision Avoidance for Distributed LTE Vehicle to Vehicle Broadcast Communications , 2018, IEEE Communications Letters.

[27]  Ching-Yao Chan,et al.  A Reinforcement Learning Based Approach for Automated Lane Change Maneuvers , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).