TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction

Data-driven simulation has become a favorable way to train and test autonomous driving algorithms. The idea of replacing the actual environment with a learned simulator has also been explored in model-based reinforcement learning in the context of world models. In this work, we show data-driven traffic simulation can be formulated as a world model. We present TrafficBots, a multi-agent policy built upon motion prediction and end-to-end driving, and based on TrafficBots we obtain a world model tailored for the planning module of autonomous vehicles. Existing data-driven traffic simulators are lacking configurability and scalability. To generate configurable behaviors, for each agent we introduce a destination as navigational information, and a time-invariant latent personality that specifies the behavioral style. To improve the scalability, we present a new scheme of positional encoding for angles, allowing all agents to share the same vectorized context and the use of an architecture based on dot-product attention. As a result, we can simulate all traffic participants seen in dense urban scenarios. Experiments on the Waymo open motion dataset show TrafficBots can simulate realistic multi-agent behaviors and achieve good performance on the motion prediction task.

[1]  Alex Kuefler,et al.  Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulation , 2022, 2022 International Conference on Robotics and Automation (ICRA).

[2]  J. L. Vázquez,et al.  Deep Interactive Motion Prediction and Planning: Playing Games with Motion Prediction Models , 2022, L4DC.

[3]  Benjamin Sapp,et al.  MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction , 2021, 2022 International Conference on Robotics and Automation (ICRA).

[4]  Igor Gilitschenski,et al.  VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles , 2021, 2022 International Conference on Robotics and Automation (ICRA).

[5]  Ollin Boer Bohan,et al.  PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulation , 2021, 2022 International Conference on Robotics and Automation (ICRA).

[6]  Samira Ebrahimi Kahou,et al.  Latent Variable Sequential Set Transformers for Joint Multi-Agent Motion Prediction , 2021, ICLR.

[7]  Jonathon Shlens,et al.  Scene Transformer: A unified architecture for predicting future trajectories of multiple agents , 2022, ICLR.

[8]  Oliver Scheel,et al.  Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients , 2021, CoRL.

[9]  Hang Zhao,et al.  DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[10]  Luc Van Gool,et al.  End-to-End Urban Driving by Imitating a Reinforcement Learning Coach , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[11]  Eric M. Wolff,et al.  Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversals , 2021, CoRL.

[12]  Eric M. Wolff,et al.  nuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles , 2021, ArXiv.

[13]  Oliver Scheel,et al.  SimNet: Learning Reactive Self-driving Simulations from Real-world Observations , 2021, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Sanja Fidler,et al.  DriveGAN: Towards a Controllable High-Quality Neural Simulation , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Jiquan Ngiam,et al.  Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[16]  Bolei Zhou,et al.  Multimodal Motion Prediction with Stacked Transformers , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Stefano V. Albrecht,et al.  GRIT: Fast, Interpretable, and Verifiable Goal Recognition with Learned Decision Trees for Autonomous Driving , 2021, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[18]  Raquel Urtasun,et al.  TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Mohammad Norouzi,et al.  Mastering Atari with Discrete World Models , 2020, ICLR.

[20]  Stefano V. Albrecht,et al.  Interpretable Goal-based Prediction and Planning for Autonomous Driving , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[21]  David Wu,et al.  AIR2 for Interaction Prediction , 2021, ArXiv.

[22]  Dong Chen,et al.  SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving , 2020, ArXiv.

[23]  Yi Shen,et al.  TNT: Target-driveN Trajectory Prediction , 2020, CoRL.

[24]  Sergio Casas,et al.  Implicit Latent Variable Model for Scene-Consistent Motion Forecasting , 2020, ECCV.

[25]  Dragomir Anguelov,et al.  VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Pratul P. Srinivasan,et al.  NeRF , 2020, ECCV.

[27]  Tie-Yan Liu,et al.  On Layer Normalization in the Transformer Architecture , 2020, ICML.

[28]  Daniela Rus,et al.  Learning Robust Control Policies for End-to-End Autonomous Driving From Data-Driven Simulation , 2020, IEEE Robotics and Automation Letters.

[29]  Marco Pavone,et al.  Trajectron++: Dynamically-Feasible Trajectory Forecasting with Heterogeneous Data , 2020, ECCV.

[30]  Jimmy Ba,et al.  Dream to Control: Learning Behaviors by Latent Imagination , 2019, ICLR.

[31]  Ruslan Salakhutdinov,et al.  Multiple Futures Prediction , 2019, NeurIPS.

[32]  Benjamin Sapp,et al.  MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction , 2019, CoRL.

[33]  Sergey Levine,et al.  PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[34]  Yann LeCun,et al.  Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic , 2019, ICLR.

[35]  Ruben Villegas,et al.  Learning Latent Dynamics for Planning from Pixels , 2018, ICML.

[36]  Xi Chen,et al.  Learning From Demonstration in the Wild , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[37]  Marco Pavone,et al.  The Trajectron: Probabilistic Multi-Agent Trajectory Modeling With Dynamic Spatiotemporal Graphs , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[38]  Yun-Pang Flötteröd,et al.  Microscopic Traffic Simulation using SUMO , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[39]  Paul Vernaza,et al.  r2p2: A ReparameteRized Pushforward Policy for Diverse, Precise Generative Path Forecasting , 2018, ECCV.

[40]  Jürgen Schmidhuber,et al.  Recurrent World Models Facilitate Policy Evolution , 2018, NeurIPS.

[41]  Germán Ros,et al.  CARLA: An Open Urban Driving Simulator , 2017, CoRL.

[42]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[43]  Philip H. S. Torr,et al.  DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Stefano Ermon,et al.  Generative Adversarial Imitation Learning , 2016, NIPS.

[45]  Honglak Lee,et al.  Learning Structured Output Representation using Deep Conditional Generative Models , 2015, NIPS.

[46]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.