Pedestrian Trajectory Prediction by Modeling the Interactions using Social LSTM Extensions

In many driving situations, human mobility is an important topic in trajectory prediction. Considering the pedestrian trajectory as a sequence generative task, a prediction algorithm based on Social Long Short-Term Memory (Social LSTM) is implemented. In order to simulate the social interaction between pedestrians, Social Pooling (S-Pooling) is used to aggregate the hidden state of pedestrians, while the attention mechanism is utilized to aggregate information differently according to the importance of surrounding pedestrians. Furthermore, Convolutional Neural Networks (CNN) is introduced into Social LSTM model to consider both the interaction between people and the characteristic of scene scale in the prediction process. Experiments are carried out against baseline methods, and the results demonstrated that combining Social LSTM with attention mechanism or CNN can improve the performance of pedestrian trajectory prediction.

[1]  A. Tordeux,et al.  Review of Pedestrian Trajectory Prediction Methods: Comparing Deep Learning and Knowledge-Based Approaches , 2021, IEEE Transactions on Intelligent Transportation Systems.

[2]  Shi-Min Hu,et al.  Beyond Self-Attention: External Attention Using Two Linear Layers for Visual Tasks , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Siew-Kei Lam,et al.  Self-Growing Spatial Graph Network for Context-Aware Pedestrian Trajectory Prediction , 2020, 2021 IEEE International Conference on Image Processing (ICIP).

[4]  S. Gelly,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2020, ICLR.

[5]  Xiong Dan,et al.  Spatial-Temporal Block and LSTM Network for Pedestrian Trajectories Prediction , 2020, ArXiv.

[6]  R. Cucchiara,et al.  DAG-Net: Double Attentive Graph Neural Network for Trajectory Forecasting , 2020, 2020 25th International Conference on Pattern Recognition (ICPR).

[7]  Zhaoxin Li,et al.  STGAT: Modeling Spatial-Temporal Interactions for Human Trajectory Prediction , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[8]  Gita Alaghband,et al.  Scene-LSTM: A Model for Human Trajectory Prediction , 2018, ArXiv.

[9]  Kwangjin Yoon,et al.  Online Multi-Object Tracking with Historical Appearance Matching and Scene Adaptive Detection Filtering , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[10]  Mark Reynolds,et al.  SS-LSTM: A Hierarchical LSTM Model for Pedestrian Trajectory Prediction , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[11]  Sridha Sridharan,et al.  Soft + Hardwired Attention: An LSTM Framework for Human Trajectory Prediction and Abnormal Event Detection , 2017, Neural Networks.

[12]  Xiaogang Wang,et al.  Pedestrian Behavior Understanding and Prediction with Deep Neural Networks , 2016, ECCV.

[13]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Robert Fitch,et al.  Bayesian intention inference for trajectory prediction with an unknown goal destination , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[15]  Alex Graves.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[16]  Luc Van Gool,et al.  You'll never walk alone: Modeling social behavior for multi-target tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  Dani Lischinski,et al.  Crowds by Example , 2007, Comput. Graph. Forum.

[18]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.