Relation-based Motion Prediction using Traffic Scene Graphs

Representing relevant information of a traffic scene and understanding its environment is crucial for the success of autonomous driving. Modeling the surrounding of an autonomous car using semantic relations, i.e., how different traffic participants relate in the context of traffic rule based behaviors, is hardly been considered in previous work. This stems from the fact that these relations are hard to extract from real-world traffic scenes. In this work, we model traffic scenes in a form of spatial semantic scene graphs for various different predictions about the traffic participants, e.g., acceleration and deceleration. Our learning and inference approach uses Graph Neural Networks (GNNs) and shows that incorporating explicit information about the spatial semantic relations between traffic participants improves the predicdtion results. Specifically, the acceleration prediction of traffic participants is improved by up to 12% compared to the baselines, which do not exploit this explicit information. Furthermore, by including additional information about previous scenes, we achieve 73% improvements.

[1]  J. M. Zöllner,et al.  Towards Traffic Scene Description: The Semantic Scene Graph , 2021, 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC).

[2]  Masayoshi Tomizuka,et al.  Spatio-Temporal Graph Dual-Attention Network for Multi-Agent Prediction and Tracking , 2021, IEEE Transactions on Intelligent Transportation Systems.

[3]  Omar Y. Al-Jarrah,et al.  Deep Learning-based Vehicle Behaviour Prediction For Autonomous Driving Applications: A Review , 2019, ArXiv.

[4]  Masayoshi Tomizuka,et al.  Multi-Agent Driving Behavior Prediction across Different Scenarios with Self-Supervised Domain Knowledge , 2021, 2021 IEEE International Intelligent Transportation Systems Conference (ITSC).

[5]  Kun Jiang,et al.  PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving , 2021, 2021 IEEE International Intelligent Transportation Systems Conference (ITSC).

[6]  Chen Lv,et al.  Graph and Recurrent Neural Network-based Vehicle Trajectory Prediction For Highway Driving , 2021, 2021 IEEE International Intelligent Transportation Systems Conference (ITSC).

[7]  Mingyu Fan,et al.  AST-GNN: An attention-based spatio-temporal graph neural network for Interaction-aware pedestrian trajectory prediction , 2021, Neurocomputing.

[8]  Lavdim Halilaj,et al.  A Knowledge Graph-Based Approach for Situation Comprehension in Driving Scenarios , 2021, ESWC.

[9]  Shuai Yi,et al.  Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction , 2020, ECCV.

[10]  Dragomir Anguelov,et al.  VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Benjamin Sapp,et al.  MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction , 2019, CoRL.

[12]  Fang Fang,et al.  Ontology-based Reasoning Approach for Long-term Behavior Prediction of Road Users , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[13]  Masayoshi Tomizuka,et al.  INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps , 2019, ArXiv.

[14]  Florin Leon,et al.  A Review of Tracking, Prediction and Decision Making Methods for Autonomous Driving , 2019, Mathematics.

[15]  Aynaz Taheri,et al.  Predictive Temporal Embedding of Dynamic Graphs , 2019, 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM).

[16]  M. Chuah,et al.  GRIP++: Enhanced Graph-based Interaction-aware Trajectory Prediction for Autonomous Driving , 2019 .

[17]  Mooi Choo Chuah,et al.  GRIP: Graph-based Interaction-aware Trajectory Prediction , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[18]  Benjamin Sapp,et al.  Rules of the Road: Predicting Driving Behavior With a Convolutional Model of Semantic Interactions , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Alois Knoll,et al.  Graph Neural Networks for Modelling Traffic Participant Interaction , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[20]  Siew Kei Lam,et al.  Situation-Aware Pedestrian Trajectory Prediction with Spatio-Temporal Attention Model , 2019, ArXiv.

[21]  Mayank Bansal,et al.  ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst , 2018, Robotics: Science and Systems.

[22]  Antonios Karatzoglou,et al.  Using a Knowledge Graph of Scenes to Enable Search of Autonomous Driving Data , 2019, SEMWEB.

[23]  Sergio Casas,et al.  IntentNet: Learning to Predict Intention from Raw Sensor Data , 2018, CoRL.

[24]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[25]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[26]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[27]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[28]  Markus Maurer,et al.  Graph-based context representation, environment modeling and information aggregation for automated driving , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[29]  Martin Lauer,et al.  3D Traffic Scene Understanding From Movable Platforms , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Julius Ziegler,et al.  Optimal trajectory generation for dynamic street scenarios in a Frenét Frame , 2010, 2010 IEEE International Conference on Robotics and Automation.

[31]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[32]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.