Integrated Graphical Representation of Highway Scenarios to Improve Trajectory Prediction of Surrounding Vehicles

Varying numbers and types of vehicles, various road structures and traffic rules bring difficulties to an autonomous vehicle driving in highway traffic scenarios. It is important to simultaneously consider all these elements in an integrated framework when predicting the future trajectories of surrounding vehicles. This paper presents a unified graphical representation method for dynamic traffic scenarios based on integrating not only the constraints from vehicles but also the collision risk implied behind road structures and traffic rules. Different from previous studies which ignores road structures and traffic rules or separately represent them in a qualitative way, this method can make better use of the influences of these environment elements in a quantitative way to improve trajectory prediction of surrounding vehicles in highway scenarios. Specifically, this method first assesses the permissibility and passability of different types of roads and vehicles, and then defines their collision risk level based on traffic rules like speed limit or priority signs. Secondly, the bird-view scenarios are rasterized and rendered as color-scaled feature maps based on this risk level. Finally, multiple feature maps with various spatial and temporal perspective field are sequentially fed to a convolutional long short-term memory (LSTM) network to predict trajectories for surrounding vehicles. We implement the model on NGSIM dataset and achieve satisfying prediction accuracy, for example, the error over 5s time prediction horizon can be reduced to less than 2.78 m in longitudinal and 0.26 m in lateral direction.

[1]  Kunpeng Zhang,et al.  AI-TP: Attention-Based Interaction-Aware Trajectory Prediction for Autonomous Driving , 2023, IEEE Transactions on Intelligent Vehicles.

[2]  Lin Zhang,et al.  A Survey on Trajectory-Prediction Methods for Autonomous Driving , 2022, IEEE Transactions on Intelligent Vehicles.

[3]  Jean-Luc Starck,et al.  Results of the 2020 fastMRI Challenge for Machine Learning MR Image Reconstruction , 2021, IEEE Transactions on Medical Imaging.

[4]  Kimihiko Nakano,et al.  Evaluation of Driver Assistance System Presenting Information of Other Vehicles through Peripheral Vision at Unsignalized Intersections , 2020, International Journal of Intelligent Transportation Systems Research.

[5]  Wenjun Wang,et al.  Interactive Trajectory Prediction of Surrounding Road Users for Autonomous Driving Using Structural-LSTM Network , 2020, IEEE Transactions on Intelligent Transportation Systems.

[6]  Kimihiko Nakano,et al.  Modeling and analysis of driver behavior under shared control through weighted visual and haptic guidance , 2020, IET Intelligent Transport Systems.

[7]  Bo Cheng,et al.  Indirect Shared Control for Cooperative Driving Between Driver and Automation in Steer-by-Wire Vehicles , 2020, IEEE Transactions on Intelligent Transportation Systems.

[8]  Dragomir Anguelov,et al.  VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Anne Verroust-Blondet,et al.  Attention Based Vehicle Trajectory Prediction , 2020, IEEE Transactions on Intelligent Vehicles.

[10]  I. Mezić,et al.  Data-driven analysis and forecasting of highway traffic dynamics , 2020, Nature Communications.

[11]  Keqiang Li,et al.  Pedestrian Trajectory Prediction Combining Probabilistic Reasoning and Sequence Learning , 2020, IEEE Transactions on Intelligent Vehicles.

[12]  Benjamin Sapp,et al.  MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction , 2019, CoRL.

[13]  Tim Fingscheidt,et al.  Analysis of the Effect of Various Input Representations for LSTM-Based Trajectory Prediction , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[14]  M. Chuah,et al.  GRIP++: Enhanced Graph-based Interaction-aware Trajectory Prediction for Autonomous Driving , 2019 .

[15]  Mooi Choo Chuah,et al.  GRIP: Graph-based Interaction-aware Trajectory Prediction , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[16]  Benjamin Sapp,et al.  Rules of the Road: Predicting Driving Behavior With a Convolutional Model of Semantic Interactions , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Dariu M. Gavrila,et al.  Context-based cyclist path prediction using Recurrent Neural Networks , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[18]  Wei Zhan,et al.  Multi-modal Probabilistic Prediction of Interactive Behavior via an Interpretable Model , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[19]  Dinesh Manocha,et al.  TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents , 2018, AAAI.

[20]  Jianyu Chen,et al.  Intention-aware Long Horizon Trajectory Prediction of Surrounding Vehicles using Dual LSTM Networks , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[21]  Sergio Casas,et al.  IntentNet: Learning to Predict Intention from Raw Sensor Data , 2018, CoRL.

[22]  Henggang Cui,et al.  Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[23]  J. Schneider,et al.  Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Driving , 2018, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[24]  Mohan M. Trivedi,et al.  Convolutional Social Pooling for Vehicle Trajectory Prediction , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  Chung Choo Chung,et al.  Sequence-to-Sequence Prediction of Vehicle Trajectory via LSTM Encoder-Decoder Architecture , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[26]  Florent Altché,et al.  An LSTM network for highway trajectory prediction , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[27]  Mykel J. Kochenderfer,et al.  Analysis of Recurrent Neural Networks for Probabilistic Modeling of Driver Behavior , 2017, IEEE Transactions on Intelligent Transportation Systems.

[28]  Xiaogang Wang,et al.  Pedestrian Behavior Understanding and Prediction with Deep Neural Networks , 2016, ECCV.

[29]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[31]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[32]  Ulrich Kressel,et al.  Probabilistic trajectory prediction with Gaussian mixture models , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[33]  Amaury Nègre,et al.  Probabilistic Analysis of Dynamic Scenes and Collision Risks Assessment to Improve Driving Safety , 2011, IEEE Intelligent Transportation Systems Magazine.

[34]  Fawzi Nashashibi,et al.  Real time trajectory prediction for collision risk estimation between vehicles , 2009, 2009 IEEE 5th International Conference on Intelligent Computer Communication and Processing.

[35]  K.C.J. Dietmayer,et al.  IMM object tracking for high dynamic driving maneuvers , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[36]  Eamonn J. Keogh,et al.  Scaling up dynamic time warping for datamining applications , 2000, KDD '00.

[37]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.