HYPER: Learned Hybrid Trajectory Prediction via Factored Inference and Adaptive Sampling

Modeling multi-modal high-level intent is important for ensuring diversity in trajectory prediction. Existing approaches explore the discrete nature of human intent before predicting continuous trajectories, to improve accuracy and support explainability. However, these approaches often assume the intent to remain fixed over the prediction horizon, which is problematic in practice, especially over longer horizons. To overcome this limitation, we introduce HYPER, a general and expressive hybrid prediction framework that models evolving human intent. By modeling traffic agents as a hybrid discretecontinuous system, our approach is capable of predicting discrete intent changes over time. We learn the probabilistic hybrid model via a maximum likelihood estimation problem and leverage neural proposal distributions to sample adaptively from the exponentially growing discrete space. The overall approach affords a better trade-off between accuracy and coverage. We train and validate our model on the Argoverse dataset, and demonstrate its effectiveness through comprehensive ablation studies and comparisons with state-of-the-art models.

[1]  Truyen Tran,et al.  Goal-driven Long-Term Trajectory Prediction , 2020, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[2]  Dragomir Anguelov,et al.  VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Marco Pavone,et al.  Trajectron++: Multi-Agent Generative Trajectory Forecasting With Heterogeneous Data for Control , 2020, ArXiv.

[4]  Elena Corina Grigore,et al.  CoverNet: Multimodal Behavior Prediction Using Trajectory Sets , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Brian C. Williams,et al.  Hybrid estimation of complex systems , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  Philip H. S. Torr,et al.  DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Jun Won Choi,et al.  LaPred: Lane-Aware Prediction of Multi-Modal Future Trajectories of Dynamic Agents , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Brian C. Williams,et al.  Best-First Enumeration Based on Bounding Conflicts, and its Application to Large-scale Hybrid Estimation , 2020, J. Artif. Intell. Res..

[9]  Gustav Markkula,et al.  Maneuver-based Anchor Trajectory Hypotheses at Roundabouts , 2021, ArXiv.

[10]  Xin Huang,et al.  DiversityGAN: Diversity-Aware Vehicle Motion Prediction via Latent Semantic Sampling , 2020, IEEE Robotics and Automation Letters.

[11]  Richard E. Turner,et al.  Neural Adaptive Sequential Monte Carlo , 2015, NIPS.

[13]  Drakoulis Richardos,et al.  Vehicle maneuver-based long-term trajectory prediction at intersection crossings , 2020, 2020 IEEE 3rd Connected and Automated Vehicles Symposium (CAVS).

[14]  Kris Kitani,et al.  PTP: Parallelized Tracking and Prediction With Graph Neural Networks and Diversity Sampling , 2021, IEEE Robotics and Automation Letters.

[15]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[16]  Marco Pavone,et al.  The Trajectron: Probabilistic Multi-Agent Trajectory Modeling With Dynamic Spatiotemporal Graphs , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[17]  Teofilo F. GONZALEZ,et al.  Clustering to Minimize the Maximum Intercluster Distance , 1985, Theor. Comput. Sci..

[18]  Alexandre Alahi,et al.  Interpretable Social Anchors for Human Trajectory Forecasting in Crowds , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Jongeun Choi,et al.  Sequential Bayesian Prediction and Adaptive Sampling Algorithms for Mobile Sensor Networks , 2012, IEEE Transactions on Automatic Control.

[20]  Peter Andersson,et al.  Adaptive Forgetting in Recursive Identification through Multiple Models , 1985 .

[21]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[22]  Chung Choo Chung,et al.  Sequence-to-Sequence Prediction of Vehicle Trajectory via LSTM Encoder-Decoder Architecture , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[23]  Bolei Zhou,et al.  Multimodal Motion Prediction with Stacked Transformers , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Simon Lucey,et al.  Argoverse: 3D Tracking and Forecasting With Rich Maps , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Yang An,et al.  From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting , 2020, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[26]  Brian C. Williams,et al.  A combined stochastic and greedy hybrid estimation capability for concurrent hybrid models with autonomous mode transitions , 2008, Robotics Auton. Syst..

[27]  Arnaud Doucet,et al.  An adaptive sequential Monte Carlo method for approximate Bayesian computation , 2011, Statistics and Computing.

[28]  Y. Bar-Shalom,et al.  The interacting multiple model algorithm for systems with Markovian switching coefficients , 1988 .

[29]  Thomas Brox,et al.  Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Fabien Moutarde,et al.  HOME: Heatmap Output for future Motion Estimation , 2021, 2021 IEEE International Intelligent Transportation Systems Conference (ITSC).

[31]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Silvio Savarese,et al.  Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  Ben Poole,et al.  Categorical Reparameterization with Gumbel-Softmax , 2016, ICLR.

[34]  Xin Huang,et al.  Fast Risk Assessment for Autonomous Vehicles Using Learned Models of Agent Futures , 2020, Robotics: Science and Systems.

[35]  Scott W. Linderman,et al.  Bayesian Learning and Inference in Recurrent Switching Linear Dynamical Systems , 2017, AISTATS.

[36]  Dragomir Anguelov,et al.  A General Algorithm for Approximate Inference and Its Application to Hybrid Bayes Nets , 1999, UAI.

[37]  Luca Anthony Thiede,et al.  Analyzing the Variety Loss in the Context of Probabilistic Trajectory Prediction , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[38]  Arnaud Doucet,et al.  Towards scaling up Markov chain Monte Carlo: an adaptive subsampling approach , 2014, ICML.

[39]  Mohan M. Trivedi,et al.  Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMs , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[40]  Dawn M. Tilbury,et al.  Multimodal Hybrid Pedestrian: A Hybrid Automaton Model of Urban Pedestrian Behavior for Automated Driving Applications , 2021, IEEE Access.

[41]  J. Malik,et al.  It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction , 2020, ECCV.

[42]  Patrick van der Smagt,et al.  Switching Linear Dynamics for Variational Bayes Filtering , 2019, ICML.

[43]  Scott W. Linderman,et al.  Variational Sequential Monte Carlo , 2017, AISTATS.

[44]  Yi Shen,et al.  TNT: Target-driveN Trajectory Prediction , 2020, CoRL.

[45]  Benjamin Sapp,et al.  MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction , 2019, CoRL.

[46]  Rui Zhao,et al.  Maximum Entropy-Regularized Multi-Goal Reinforcement Learning , 2019, ICML.

[47]  Nicholas Rhinehart,et al.  Generative Hybrid Representations for Activity Forecasting With No-Regret Learning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Buyu Liu,et al.  Divide-and-Conquer for Lane-Aware Diverse Trajectory Prediction , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Uri Shalit,et al.  Structured Inference Networks for Nonlinear State Space Models , 2016, AAAI.

[50]  Brian C. Williams,et al.  Model learning for switching linear systems with autonomous mode transitions , 2007, 2007 46th IEEE Conference on Decision and Control.

[51]  Brian C. Williams,et al.  Mode Estimation of Probabilistic Hybrid Systems , 2002, HSCC.

[52]  Ming Jian Zuo,et al.  A new adaptive sequential sampling method to construct surrogate models for efficient reliability analysis , 2018, Reliab. Eng. Syst. Saf..

[53]  Qifeng Chen,et al.  Learning to Predict Vehicle Trajectories with Model-based Planning , 2021, CoRL.

[54]  Kris Kitani,et al.  Diverse Trajectory Forecasting with Determinantal Point Processes , 2019, ICLR.