A Novel Graph-Based Trajectory Predictor With Pseudo-Oracle

Pedestrian trajectory prediction in dynamic scenes remains a challenging and critical problem in numerous applications, such as self-driving cars and socially aware robots. Challenges concentrate on capturing pedestrians' motion patterns and social interactions, as well as handling the future uncertainties. Recent studies focus on modeling pedestrians' motion patterns with recurrent neural networks, capturing social interactions with pooling- or graph-based methods, and handling future uncertainties by using the random Gaussian noise as the latent variable. However, they do not integrate specific obstacle avoidance experiences (OAEs) that may improve prediction performance. For example, pedestrians' future trajectories are always influenced by others in front. Here, we propose the Graph-based Trajectory Predictor with Pseudo-Oracle (GTPPO), an encoder-decoder-based method conditioned on pedestrians' future behaviors. Pedestrians' motion patterns are encoded with a long short-term memory unit, which introduces temporal attention to highlight specific time steps. Their interactions are captured by a graph-based attention mechanism, which draws OAE into the data-driven learning process of graph attention. Future uncertainties are handled by generating multimodal outputs with an informative latent variable. Such a variable is generated by a novel pseudo-oracle predictor, which minimizes the knowledge gap between historical and ground-truth trajectories. Finally, the GTPPO is evaluated on ETH, UCY, and Stanford Drone datasets, and the results demonstrate state-of-the-art performance. Besides, the qualitative evaluations show successful cases of handling sudden motion changes in the future. Such findings indicate that GTPPO can peek into the future.

[1]  Nanning Zheng,et al.  SR-LSTM: State Refinement for LSTM Towards Pedestrian Trajectory Prediction , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Wolfgang Rosenstiel,et al.  Object-oriented Bayesian networks for detection of lane change maneuvers , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[3]  Junqiang Xi,et al.  Learning and Inferring a Driver's Braking Action in Car-Following Scenarios , 2018, IEEE Transactions on Vehicular Technology.

[4]  Zhaoxin Li,et al.  STGAT: Modeling Spatial-Temporal Interactions for Human Trajectory Prediction , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[5]  Shiri Gordon,et al.  An efficient image similarity measure based on approximations of KL-divergence between two gaussian mixtures , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[6]  Ruslan Salakhutdinov,et al.  Multiple Futures Prediction , 2019, NeurIPS.

[7]  Kris Kitani,et al.  Diverse Trajectory Forecasting with Determinantal Point Processes , 2019, ICLR.

[8]  Li-Ta Hsu,et al.  Human-like motion planning model for driving in signalized intersections , 2017 .

[9]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[10]  Peng Wang,et al.  Appearance based pedestrians' head pose and body orientation estimation using deep learning , 2018, Neurocomputing.

[11]  Wei Zhan,et al.  Generic Vehicle Tracking Framework Capable of Handling Occlusions Based on Modified Mixture Particle Filter , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[12]  Denis Wolf,et al.  Scene Compliant Trajectory Forecast With Agent-Centric Spatio-Temporal Grids , 2019, IEEE Robotics and Automation Letters.

[13]  Silvio Savarese,et al.  Learning Social Etiquette: Human Trajectory Understanding In Crowded Scenes , 2016, ECCV.

[14]  Julien Pettré,et al.  Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories With GANs , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[15]  Dinesh Manocha,et al.  TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents , 2018, AAAI.

[16]  Dongchun Ren,et al.  StarNet: Pedestrian Trajectory Prediction using Deep Neural Network in Star Topology , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Silvio Savarese,et al.  SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Marco Cristani,et al.  Transformer Networks for Trajectory Forecasting , 2020, 2020 25th International Conference on Pattern Recognition (ICPR).

[19]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[20]  Silvio Savarese,et al.  Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks , 2019, NeurIPS.

[21]  J. Malik,et al.  It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction , 2020, ECCV.

[22]  Haoran Song,et al.  PiP: Planning-informed Trajectory Prediction for Autonomous Driving , 2020, ECCV.

[23]  Yue Hu,et al.  Collaborative Motion Prediction via Neural Motion Message Passing , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Mark Reynolds,et al.  SS-LSTM: A Hierarchical LSTM Model for Pedestrian Trajectory Prediction , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[25]  Xiaogang Wang,et al.  Understanding pedestrian behaviors from stationary crowd groups , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Alessio Del Bue,et al.  "Seeing is Believing": Pedestrian Trajectory Forecasting Using Visual Frustum of Attention , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[27]  Marco Pavone,et al.  The Trajectron: Probabilistic Multi-Agent Trajectory Modeling With Dynamic Spatiotemporal Graphs , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[28]  S. Savarese,et al.  Learning an Image-Based Motion Context for Multiple People Tracking , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Luc Van Gool,et al.  Improving Data Association by Joint Modeling of Pedestrian Trajectories and Groupings , 2010, ECCV.

[30]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[31]  Amaury Nègre,et al.  Probabilistic Analysis of Dynamic Scenes and Collision Risks Assessment to Improve Driving Safety , 2011, IEEE Intelligent Transportation Systems Magazine.

[32]  Kai Oliver Arras,et al.  People tracking with human motion predictions from social forces , 2010, 2010 IEEE International Conference on Robotics and Automation.

[33]  Brendan Tran Morris,et al.  SSeg-LSTM: Semantic Scene Segmentation for Trajectory Prediction , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[34]  Lionel Ott,et al.  Spatiotemporal Learning of Directional Uncertainty in Urban Environments With Kernel Recurrent Mixture Density Networks , 2019, IEEE Robotics and Automation Letters.

[35]  Lamberto Ballan,et al.  Social and Scene-Aware Trajectory Prediction in Crowded Spaces , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[36]  Claudia Blaiotta,et al.  Learning Generative Socially Aware Models of Pedestrian Motion , 2019, IEEE Robotics and Automation Letters.

[37]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Silvio Savarese,et al.  Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[39]  Philip H. S. Torr,et al.  DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Srikanth Malla,et al.  DROGON: A Causal Reasoning Framework for Future Trajectory Forecast , 2019, ArXiv.

[41]  Rob Fergus,et al.  Stochastic Video Generation with a Learned Prior , 2018, ICML.

[42]  Siew Kei Lam,et al.  Situation-Aware Pedestrian Trajectory Prediction with Spatio-Temporal Attention Model , 2019, ArXiv.

[43]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[44]  Michael Goldhammer,et al.  Trajectory prediction of cyclists using a physical model and an artificial neural network , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[45]  Ping Guo,et al.  Stochastic trajectory prediction with social graph network , 2019, ArXiv.

[46]  Masayoshi Tomizuka,et al.  Wasserstein Generative Learning with Kinematic Constraints for Probabilistic Interactive Driving Behavior Prediction , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[47]  Efstratios Gavves,et al.  SafeCritic: Collision-Aware Trajectory Prediction , 2019, ArXiv.

[48]  Marco Pavone,et al.  Trajectron++: Multi-Agent Generative Trajectory Forecasting With Heterogeneous Data for Control , 2020, ArXiv.

[49]  Grzegorz Cielniak,et al.  Modelling and Predicting Rhythmic Flow Patterns in Dynamic Environments , 2018, TAROS.

[50]  Sridha Sridharan,et al.  Soft + Hardwired Attention: An LSTM Framework for Human Trajectory Prediction and Abnormal Event Detection , 2017, Neural Networks.

[51]  N. Roy,et al.  Mobile Agent Trajectory Prediction using Bayesian Nonparametric Reachability Trees , 2011 .

[52]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[53]  Benjamin Sapp,et al.  Rules of the Road: Predicting Driving Behavior With a Convolutional Model of Semantic Interactions , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).