GOHOME: Graph-Oriented Heatmap Output for future Motion Estimation

In this paper, we propose GOHOME, a method leveraging graph representations of the High Definition Map and sparse projections to generate a heatmap output representing the future position probability distribution for a given agent in a traffic scene. This heatmap output yields an unconstrained 2D grid representation of agent future possible locations, allowing inherent multimodality and a measure of the uncertainty of the prediction. Our graph-oriented model avoids the high computation burden of representing the surrounding context as squared images and processing it with classical CNNs, but focuses instead only on the most probable lanes where the agent could end up in the immediate future. GOHOME reaches 2nd on Argoverse Motion Forecasting Benchmark on the MissRate6 metric while achieving significant speed-up and memory burden diminution compared to Argoverse 1 place method HOME. We also highlight that heatmap output enables multimodal ensembling and improve 1 place MissRate6 by more than 15% with our best ensemble on Argoverse. Finally, we evaluate and reach state-of-the-art performance on the other trajectory prediction datasets nuScenes and Interaction, demonstrating the generalizability of our method.

[1]  Simon Lucey,et al.  Argoverse: 3D Tracking and Forecasting With Rich Maps , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Mohan M. Trivedi,et al.  Trajectory Forecasts in Unknown Environments Conditioned on Grid-Based Plans , 2020, ArXiv.

[3]  Chen Lv,et al.  Heterogeneous Edge-Enhanced Graph Attention Network For Multi-Agent Trajectory Prediction , 2021, ArXiv.

[4]  Yang An,et al.  From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting , 2020, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[5]  Qifeng Chen,et al.  Learning to Predict Vehicle Trajectories with Model-based Planning , 2021, CoRL.

[6]  Mohan M. Trivedi,et al.  Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMs , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[7]  Geoffrey E. Hinton,et al.  Layer Normalization , 2016, ArXiv.

[8]  Chen Lv,et al.  ReCoG: A Deep Learning Framework with Heterogeneous Graph for Interaction-Aware Trajectory Prediction , 2020, ArXiv.

[9]  Florent Altché,et al.  An LSTM network for highway trajectory prediction , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[10]  Qiang Xu,et al.  nuScenes: A Multimodal Dataset for Autonomous Driving , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  David J. Crandall,et al.  Stepwise Goal-Driven Networks for Trajectory Prediction , 2021, IEEE Robotics and Automation Letters.

[12]  Dragomir Anguelov,et al.  VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Jiquan Ngiam,et al.  Scene Transformer: A unified multi-task model for behavior prediction and planning , 2021, ArXiv.

[14]  Yi Shen,et al.  TNT: Target-driveN Trajectory Prediction , 2020, CoRL.

[15]  Qifeng Chen,et al.  TPCN: Temporal Point Cloud Networks for Motion Forecasting , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  R. Urtasun,et al.  Learning Lane Graph Representations for Motion Forecasting , 2020, ECCV.

[17]  Ruslan Salakhutdinov,et al.  Multiple Futures Prediction , 2019, NeurIPS.

[18]  Alan Yuille,et al.  Probabilistic Multi-modal Trajectory Prediction with Lane Attention for Autonomous Vehicles , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[19]  Jun Won Choi,et al.  LaPred: Lane-Aware Prediction of Multi-Modal Future Trajectories of Dynamic Agents , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Silvio Savarese,et al.  SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Benjamin Sapp,et al.  MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction , 2019, CoRL.

[22]  J. Malik,et al.  It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction , 2020, ECCV.

[23]  Paul Vernaza,et al.  r2p2: A ReparameteRized Pushforward Policy for Diverse, Precise Generative Path Forecasting , 2018, ECCV.

[24]  Xingyi Zhou,et al.  Objects as Points , 2019, ArXiv.

[25]  Jean Pierre Mercat,et al.  Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecasting , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Buyu Liu,et al.  Divide-and-Conquer for Lane-Aware Diverse Trajectory Prediction , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  D. Ramanan,et al.  What-If Motion Prediction for Autonomous Driving , 2020, ArXiv.

[28]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[29]  Hang Zhao,et al.  DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[30]  Fabien Moutarde,et al.  Asymmetrical Bi-RNN for pedestrian trajectory encoding , 2021, ArXiv.

[31]  Renjie Liao,et al.  LaneRCNN: Distributed Representations for Graph-Centric Motion Forecasting , 2021, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[32]  Marco Pavone,et al.  Trajectron++: Dynamically-Feasible Trajectory Forecasting with Heterogeneous Data , 2020, ECCV.

[34]  Mohan M. Trivedi,et al.  Multi-Head Attention with Joint Agent-Map Representation for Trajectory Prediction in Autonomous Driving , 2020, ArXiv.

[35]  Jiquan Ngiam,et al.  Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[36]  Bolei Zhou,et al.  Multimodal Motion Prediction with Stacked Transformers , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Elena Corina Grigore,et al.  CoverNet: Multimodal Behavior Prediction Using Trajectory Sets , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Philip H. S. Torr,et al.  DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Henggang Cui,et al.  Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[40]  Frank D. Wood,et al.  Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulation , 2021, 2021 IEEE International Intelligent Transportation Systems Conference (ITSC).

[41]  Fabien Moutarde,et al.  HOME: Heatmap Output for future Motion Estimation , 2021, 2021 IEEE International Intelligent Transportation Systems Conference (ITSC).

[42]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).