ProphNet: Efficient Agent-Centric Motion Forecasting with Anchor-Informed Proposals

Motion forecasting is a key module in an autonomous driving system. Due to the heterogeneous nature of multi-sourced input, multimodality in agent behavior, and low latency required by onboard deployment, this task is notoriously challenging. To cope with these difficulties, this paper proposes a novel agent-centric model with anchor-informed proposals for efficient multimodal motion prediction. We design a modality-agnostic strategy to concisely encode the complex input in a unified manner. We generate diverse proposals, fused with anchors bearing goal-oriented scene context, to induce multimodal prediction that covers a wide range of future trajectories. Our network architecture is highly uniform and succinct, leading to an efficient model amenable for real-world driving deployment. Experiments reveal that our agent-centric network compares favorably with the state-of-the-art methods in prediction accuracy, while achieving scene-centric level inference latency.

[1]  R. Urtasun,et al.  GoRela: Go Relative for Viewpoint-Invariant Motion Forecasting , 2022, 2023 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Khaled S. Refaat,et al.  Wayformer: Motion Forecasting via Simple & Efficient Attention Networks , 2022, 2023 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Ziyao Xu,et al.  TENET: Transformer Encoding Network for Effective Temporal Flow on Motion Prediction , 2022, ArXiv.

[4]  Chen Chen,et al.  BANet: Motion Forecasting with Boundary Aware Network , 2022, ArXiv.

[5]  Qifeng Chen,et al.  Bootstrap Motion Forecasting With Self-Consistent Constraints , 2022, IEEE International Conference on Computer Vision.

[6]  Xiaodong Yang,et al.  TL-GAN: Improving Traffic Light Recognition via Data Synthesis for Autonomous Driving , 2022, ArXiv.

[7]  Fang Da,et al.  Path-Aware Graph Attention for HD Maps in Motion Prediction , 2022, 2022 International Conference on Robotics and Automation (ICRA).

[8]  Benjamin Sapp,et al.  MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction , 2021, 2022 International Conference on Robotics and Automation (ICRA).

[9]  Fabien Moutarde,et al.  GOHOME: Graph-Oriented Heatmap Output for future Motion Estimation , 2021, 2022 International Conference on Robotics and Automation (ICRA).

[10]  James Hays,et al.  Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting , 2023, NeurIPS Datasets and Benchmarks.

[11]  Alan Yuille,et al.  Exploring Simple 3D Multi-Object Tracking for Autonomous Driving , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[12]  Hang Zhao,et al.  DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[13]  Quoc V. Le,et al.  Pay Attention to MLPs , 2021, NeurIPS.

[14]  Alan Yuille,et al.  Self-Supervised Pillar Motion Learning for Autonomous Driving , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Bolei Zhou,et al.  Multimodal Motion Prediction with Stacked Transformers , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Qifeng Chen,et al.  Learning to Predict Vehicle Trajectories with Model-based Planning , 2021, CoRL.

[17]  Qifeng Chen,et al.  TPCN: Temporal Point Cloud Networks for Motion Forecasting , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Renjie Liao,et al.  LaneRCNN: Distributed Representations for Graph-Centric Motion Forecasting , 2021, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[19]  Jiquan Ngiam,et al.  Scene Transformer: A unified multi-task model for behavior prediction and planning , 2021, ArXiv.

[20]  Yi Shen,et al.  TNT: Target-driveN Trajectory Prediction , 2020, CoRL.

[21]  R. Urtasun,et al.  Learning Lane Graph Representations for Motion Forecasting , 2020, ECCV.

[22]  Dragomir Anguelov,et al.  VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Freddy A. Boulton,et al.  CoverNet: Multimodal Behavior Prediction Using Trajectory Sets , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Bingqi Zhang,et al.  High Definition Map for Automated Driving: Overview and Analysis , 2020, Journal of Navigation.

[25]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[26]  Ruslan Salakhutdinov,et al.  Multiple Futures Prediction , 2019, NeurIPS.

[27]  Benjamin Sapp,et al.  MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction , 2019, CoRL.

[28]  Simon Lucey,et al.  Argoverse: 3D Tracking and Forecasting With Rich Maps , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[30]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.