Predictive Freeway Overtaking Strategy for Automated Vehicles Using Deep Reinforcement Learning

This paper aims to combine the deep learning (DL) and reinforcement learning (RL) techniques to formulate the predictive overtaking strategy for autonomous vehicles in freeway scenario. First, the real-world driving data is extracted from the Next Generation SIMulation (NGSIM) dataset. The long short-term memory (LSTM) model is leveraged to forecast the longitudinal and lateral motion of vehicles. Then, the freeway overtaking scenario is constructed, wherein two-lane transportation is considered. Based on the predicted driving trajectories of the surrounding vehicles, RL is utilized to guide the target vehicle passes through the scenario as soon as possible. Results indicate that the presented decision-making strategy could enhance the mobility and safety of the studied automated vehicle. The availability of the proposed method in uncertain and complex driving situations is also demonstrated.

[1]  Yi Zhang,et al.  Human-like Autonomous Vehicle Speed Control by Deep Reinforcement Learning with Double Q-Learning , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[2]  Hilbert J. Kappen,et al.  Speedy Q-Learning , 2011, NIPS.

[3]  George Konidaris,et al.  Value Function Approximation in Reinforcement Learning Using the Fourier Basis , 2011, AAAI.

[4]  William Whittaker,et al.  Autonomous driving in urban environments: Boss and the Urban Challenge , 2008, J. Field Robotics.

[5]  John M. Dolan,et al.  Autonomous vehicle social behavior for highway entrance ramp management , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[6]  Andrea Vedaldi,et al.  MatConvNet: Convolutional Neural Networks for MATLAB , 2014, ACM Multimedia.

[7]  Dongpu Cao,et al.  Reinforcement Learning Optimized Look-Ahead Energy Management of a Parallel Hybrid Electric Vehicle , 2017, IEEE/ASME Transactions on Mechatronics.

[8]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[9]  Yi Yang,et al.  Two-Stream Multirate Recurrent Neural Network for Video-Based Pedestrian Reidentification , 2018, IEEE Transactions on Industrial Informatics.

[10]  Teng Liu,et al.  A Bi-Level Control for Energy Efficiency Improvement of a Hybrid Tracked Vehicle , 2018, IEEE Transactions on Industrial Informatics.

[11]  Bo Wang,et al.  Online Markov Chain-based energy management for a hybrid tracked vehicle with speedy Q-learning , 2018, Energy.

[12]  Weisong Shi,et al.  Teaching Autonomous Driving Using a Modular and Integrated Approach , 2018, 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC).

[13]  Sebastian Thrun,et al.  Junior: The Stanford entry in the Urban Challenge , 2008, J. Field Robotics.

[14]  Araz Taeihagh,et al.  Governing autonomous vehicles: emerging responses for safety, liability, privacy, cybersecurity, and industry risks , 2018, Transport Reviews.

[15]  Carl-Johan Hoel,et al.  Automated Speed and Lane Change Decision Making using Deep Reinforcement Learning , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[16]  D. Rus,et al.  Annual Review of Control , Robotics , and Autonomous Systems Planning and Decision-Making for Autonomous Vehicles , 2018 .

[17]  Hayoung Kim,et al.  Deep Q Learning Based High Level Driving Policy Determination , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[18]  Rüdiger Dillmann,et al.  Probabilistic decision-making under uncertainty for autonomous driving using continuous POMDPs , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).