Multi-Relational Pedestrian Trajectory Prediction in Complex Scenes

Pedestrian trajectory prediction has an important impact on the construction of smart cities and the popularization of autonomous vehicles. Pedestrian trajectory prediction in complex scenes is challenging because the trajectories are largely disturbed by the surrounding social environment. To better model the relationship between pedestrians and the social environment, we propose a novel Multi-Relation Network based on Long Short-Term Memory(LSTM). We first use Convolution Neural Network(CNN) and LSTM for feature extraction of scenes and pedestrians respectively. We realize the interaction between pedestrians and social environment by introducing attention mechanism. The experiment results on two public datasets, i.e. ETH and UCY, demonstrate that the prediction accuracy can be effectively improved by considering the social interactions.

[1]  Julien Pettré,et al.  Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories With GANs , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[2]  Chung Choo Chung,et al.  Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[3]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Yoshua Bengio,et al.  A Recurrent Latent Variable Model for Sequential Data , 2015, NIPS.

[6]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[7]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[8]  Yoshua Bengio,et al.  End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results , 2014, ArXiv.

[9]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[10]  Navdeep Jaitly,et al.  Towards End-To-End Speech Recognition with Recurrent Neural Networks , 2014, ICML.

[11]  Armand Joulin,et al.  Deep Fragment Embeddings for Bidirectional Image Sentence Mapping , 2014, NIPS.

[12]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[13]  Luis E. Ortiz,et al.  Who are you with and where are you going? , 2011, CVPR 2011.

[14]  Fawzi Nashashibi,et al.  Real time trajectory prediction for collision risk estimation between vehicles , 2009, 2009 IEEE 5th International Conference on Intelligent Computer Communication and Processing.

[15]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[16]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.