Impact-aware Maneuver Decision with Enhanced Perception for Autonomous Vehicle

Autonomous driving is an emerging technology that has developed rapidly over the last decade. There have been numerous interdisciplinary challenges imposed on the current transportation system by autonomous vehicles. In this paper, we conduct an algorithmic study on the autonomous vehicle decision-making process, which is a fundamental problem in the vehicle automation field and the root cause of most traffic congestion. We propose a perception-and-decision framework, called HEAD, which consists of an enHanced pErception module and a mAneuver Decision module. HEAD aims to enable the autonomous vehicle to perform safe, efficient, and comfortable maneuvers with minimal impact on other vehicles. In the enhanced perception module, a graph-based state prediction model with a strategy of phantom vehicle construction is proposed to predict the one-step future states for multiple surrounding vehicles in parallel, which deals with sensor limitations such as limited detection range and poor detection accuracy under occlusions. Then in the maneuver decision module, a deep reinforcement learning-based model is designed to learn a policy for the autonomous vehicle to perform maneuvers in continuous action space w.r.t. a parameterized action Markov decision process. A hybrid reward function takes into account aspects of safety, efficiency, comfort, and impact to guide the autonomous vehicle to make optimal maneuver decisions. Extensive experiments offer evidence that HEAD can advance the state of the art in terms of both macroscopic and microscopic effectiveness.

[1]  Mengyin Fu,et al.  Trajectory Prediction-Based Local Spatio-Temporal Navigation Map for Autonomous Driving in Dynamic Highway Environments , 2022, IEEE Transactions on Intelligent Transportation Systems.

[2]  Junkun Yan,et al.  Radar sensor network resource allocation for fused target tracking: A brief review , 2022, Inf. Fusion.

[3]  Yu Zheng,et al.  Detecting Loaded Trajectories for Hazardous Chemicals Transportation , 2022, 2022 IEEE 38th International Conference on Data Engineering (ICDE).

[4]  Kai Zeng,et al.  PATROL: A Velocity Control Framework for Autonomous Vehicle via Spatial-Temporal Reinforcement Learning , 2021, CIKM.

[5]  Max Q.-H. Meng,et al.  A Survey on Deep-Learning Approaches for Vehicle Trajectory Prediction in Autonomous Driving , 2021, 2021 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[6]  Han Su,et al.  Lane Change Scheduling for Autonomous Vehicle: A Prediction-and-Search Framework , 2021, KDD.

[7]  Jorge Cabral,et al.  Automotive LiDAR Technology: A Survey , 2021, IEEE Transactions on Intelligent Transportation Systems.

[8]  M. Trivedi,et al.  Trajectory Prediction for Autonomous Driving based on Multi-Head Attention with Joint Agent-Map Representation , 2020, 2021 IEEE Intelligent Vehicles Symposium (IV).

[9]  Anne Verroust-Blondet,et al.  Attention Based Vehicle Trajectory Prediction , 2020, IEEE Transactions on Intelligent Vehicles.

[10]  You Li,et al.  Lidar for Autonomous Driving: The Principles, Challenges, and Trends for Automotive Lidar and Perception Systems , 2020, IEEE Signal Processing Magazine.

[11]  Victor Talpaert,et al.  Deep Reinforcement Learning for Autonomous Driving: A Survey , 2020, IEEE Transactions on Intelligent Transportation Systems.

[12]  Szil'ard Aradi,et al.  Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles , 2020, IEEE Transactions on Intelligent Transportation Systems.

[13]  Omar Y. Al-Jarrah,et al.  Deep Learning-based Vehicle Behaviour Prediction For Autonomous Driving Applications: A Review , 2019, ArXiv.

[14]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[15]  Edouard Leurent,et al.  Social Attention for Autonomous Decision-Making in Dense Traffic , 2019, ArXiv.

[16]  James J. Q. Yu,et al.  Spatial-Temporal Graph Attention Networks: A Deep Learning Approach for Traffic Forecasting , 2019, IEEE Access.

[17]  Cheng Wang,et al.  GMAN: A Graph Multi-Attention Network for Traffic Prediction , 2019, AAAI.

[18]  Reza Langari,et al.  A Predictive Perception Model and Control Strategy for Collision-Free Autonomous Driving , 2019, IEEE Transactions on Intelligent Transportation Systems.

[19]  Santiago Royo,et al.  An Overview of Lidar Imaging Systems for Autonomous Vehicles , 2019, Applied Sciences.

[20]  Ning Feng,et al.  Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting , 2019, AAAI.

[21]  Silvio Savarese,et al.  Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks , 2019, NeurIPS.

[22]  Alexander Carballo,et al.  A Survey of Autonomous Driving: Common Practices and Emerging Technologies , 2019, IEEE Access.

[23]  Fawzi Nashashibi,et al.  Non-local Social Pooling for Vehicle Trajectory Prediction , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[24]  Yixin Cao,et al.  KGAT: Knowledge Graph Attention Network for Recommendation , 2019, KDD.

[25]  Craig J. Bester,et al.  Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spaces , 2019, ArXiv.

[26]  Carl-Johan Hoel,et al.  Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving , 2019, IEEE Transactions on Intelligent Vehicles.

[27]  Samyeul Noh,et al.  Decision-Making Framework for Autonomous Driving at Road Intersections: Safeguarding Against Collision, Overly Conservative Behavior, and Violation Vehicles , 2019, IEEE Transactions on Industrial Electronics.

[28]  Dimitar Filev,et al.  Autonomous Highway Driving using Deep Reinforcement Learning , 2019, 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC).

[29]  Zhiheng Li,et al.  Modeling Vehicle Interactions via Modified LSTM Models for Trajectory Prediction , 2019, IEEE Access.

[30]  Yanfang Ye,et al.  Heterogeneous Graph Attention Network , 2019, WWW.

[31]  Bo Fu,et al.  LiDAR-Camera Calibration Under Arbitrary Configurations: Observability and Methods , 2019, IEEE Transactions on Instrumentation and Measurement.

[32]  Pedro J. Navarro,et al.  A Systematic Review of Perception System and Simulators for Autonomous Vehicles Research , 2019, Sensors.

[33]  Meixin Zhu,et al.  Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving , 2019, ArXiv.

[34]  Torsten Bertram,et al.  Trajectory Prediction for Safety Critical Maneuvers in Automated Highway Driving , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[35]  Wei Qi Yan,et al.  Blind Spot Monitoring Using Deep Learning , 2018, 2018 International Conference on Image and Vision Computing New Zealand (IVCNZ).

[36]  Yun-Pang Flötteröd,et al.  Microscopic Traffic Simulation using SUMO , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[37]  Qing Wang,et al.  Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space , 2018, ArXiv.

[38]  Min Sun,et al.  Cube Padding for Weakly-Supervised Saliency Prediction in 360° Videos , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[39]  Mohan M. Trivedi,et al.  Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMs , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[40]  Mohan M. Trivedi,et al.  Convolutional Social Pooling for Vehicle Trajectory Prediction , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[41]  Chung Choo Chung,et al.  Sequence-to-Sequence Prediction of Vehicle Trajectory via LSTM Encoder-Decoder Architecture , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[42]  Jingjing Jiang,et al.  Lateral Control of an Autonomous Vehicle , 2018, IEEE Transactions on Intelligent Vehicles.

[43]  Meng Wang,et al.  Realistic Car-Following Models for Microscopic Simulation of Adaptive and Cooperative Adaptive Cruise Control Vehicles , 2017 .

[44]  Ke Zhang,et al.  A Short-Term Rainfall Prediction Model Using Multi-task Convolutional Neural Networks , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[45]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[46]  Florent Altché,et al.  An LSTM network for highway trajectory prediction , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[47]  Xin Li,et al.  Algorithm and hardware implementation for visual perception system in autonomous vehicle: A survey , 2017, Integr..

[48]  Christoph Stiller,et al.  Decision making for autonomous driving considering interaction and uncertain prediction of surrounding vehicles , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[49]  Taejoon Park,et al.  Toward Mitigating Phantom Jam Using Vehicle-to-Vehicle Communication , 2017, IEEE Transactions on Intelligent Transportation Systems.

[50]  Adam Prügel-Bennett,et al.  SVM Parameter Optimization using Grid Search and Genetic Algorithm to Improve Classification Performance , 2016 .

[51]  Peter Stone,et al.  Deep Reinforcement Learning in Parameterized Action Space , 2015, ICLR.

[52]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[53]  Pravesh Ranchod,et al.  Reinforcement Learning with Parameterized Actions , 2015, AAAI.

[54]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[55]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[56]  Steven E Shladover,et al.  Modeling cooperative and autonomous adaptive cruise control dynamic responses using experimental data , 2014 .

[57]  Giampaolo Ferraioli,et al.  3D automotive imaging radar for transportation systems monitoring , 2014, 2014 IEEE Workshop on Environmental, Energy, and Structural Monitoring Systems Proceedings.

[58]  Xiaoru Yuan,et al.  Visual Traffic Jam Analysis Based on Trajectory Data , 2013, IEEE Transactions on Visualization and Computer Graphics.

[59]  Hongbin Zha,et al.  A real-time motion planner with trajectory optimization for autonomous vehicles , 2012, 2012 IEEE International Conference on Robotics and Automation.

[60]  Y. Sugiyama,et al.  Traffic jams without bottlenecks—experimental evidence for the physical mechanism of the formation of a jam , 2008 .

[61]  D. Schrank,et al.  2001 Urban Mobility Report , 2001 .

[62]  Helbing,et al.  Congested traffic states in empirical observations and microscopic simulations , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[63]  Nasser Kehtarnavaz,et al.  A transportable neural-network approach to autonomous vehicle following , 1998 .

[64]  Kai Nagel,et al.  Two-lane traffic rules for cellular automata: A systematic approach , 1997, cond-mat/9712196.

[65]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[66]  P. Wagner,et al.  Metastable states in a microscopic model of traffic flow , 1997 .

[67]  Andy Shea,et al.  Traffic Safety and the Driver , 1992 .

[68]  Chin-Teng Lin,et al.  Jointly dampening traffic oscillations and improving energy consumption with electric, connected and automated vehicles: A reinforcement learning based approach , 2020 .

[69]  Elif Derya Übeyli,et al.  Recurrent Neural Networks , 2018 .

[70]  Junxiang Li,et al.  A Dynamic Bayesian Network for Vehicle Maneuver Prediction in Highway Driving Scenarios: Framework and Verification , 2019, Electronics.

[71]  Jakob Erdmann,et al.  SUMO’s Lane-Changing Model , 2015 .

[72]  Ira D. Jacobson,et al.  MODELS OF HUMAN COMFORT IN VEHICLE ENVIRONMENTS , 1980 .