LSTM based trajectory prediction model for cyclist utilizing multiple interactions with environment

Abstract The cyclist trajectory prediction is critical for the local path planning of autonomous vehicles. Based on the assumption that cyclist's movement is limited by its dynamics and subjected to interactions with environments, a novel LSTM based cyclist trajectory prediction model which utilizes multiple interactions with surroundings and motion feature in a unified framework is proposed. Road features describing road boundary and static obstacles are employed to address cyclist's interaction with the road. To address interactions with pedestrians, other cyclists and vehicles, object features including object attributes and relative positions are utilized. The focal attention mechanism is employed to reveal the importance of features at each time-steps. By feeding features into LSTM encoder, the movement in the next two seconds is predicted. Experiments were conducted on two datasets, and results show that the presented model outperforms the state-of-art models in most cases.

[1]  Navdeep Jaitly,et al.  Towards End-To-End Speech Recognition with Recurrent Neural Networks , 2014, ICML.

[2]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Yi Wen,et al.  Injured probability assessment in frontal pedestrian-vehicle collision counting uncertainties in pedestrian movement , 2018, Safety Science.

[4]  Rui Jiang,et al.  Stochastic multi-value cellular automata models for bicycle flow , 2004 .

[5]  Armand Joulin,et al.  Deep Fragment Embeddings for Bidirectional Image Sentence Mapping , 2014, NIPS.

[6]  Christophe Garcia,et al.  Contribution of recurrent connectionist language models in improving LSTM-based Arabic text recognition in videos , 2017, Pattern Recognit..

[7]  Silvio Savarese,et al.  Learning Social Etiquette: Human Trajectory Understanding In Crowded Scenes , 2016, ECCV.

[8]  Michael Goldhammer,et al.  Trajectory prediction of cyclists using a physical model and an artificial neural network , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[9]  Dariu Gavrila,et al.  UvA-DARE ( Digital Academic Repository ) Pedestrian Path Prediction with Recursive Bayesian Filters : A Comparative Study , 2013 .

[10]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[11]  Qingshan Liu,et al.  Visual tracking using spatio-temporally nonlocally regularized correlation filter , 2018, Pattern Recognit..

[12]  Dariu Gavrila,et al.  Using road topology to improve cyclist path prediction , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[13]  Huchuan Lu,et al.  Multi attention module for visual tracking , 2019, Pattern Recognit..

[14]  Dariu M. Gavrila,et al.  Context-based cyclist path prediction using Recurrent Neural Networks , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[15]  Shenghua Gao,et al.  Encoding Crowd Interaction with Deep Neural Network for Pedestrian Trajectory Prediction , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Yanning Zhang,et al.  Human trajectory prediction in crowded scene using social-affinity Long Short-Term Memory , 2019, Pattern Recognit..

[17]  Martin Fellendorf,et al.  Modeling Concepts for Mixed Traffic , 2012 .

[18]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[19]  Silvio Savarese,et al.  SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Prabhat Kumar,et al.  RNN based online handwritten word recognition in Devanagari and Bengali scripts using horizontal zoning , 2019, Pattern Recognit..

[21]  Juan Carlos Niebles,et al.  Peeking Into the Future: Predicting Future Person Activities and Locations in Videos , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Dariu Gavrila,et al.  Context-Based Path Prediction for Targets with Switching Dynamics , 2018, International Journal of Computer Vision.

[23]  Jian-Huang Lai,et al.  Detecting abnormal crowd behaviors based on the div-curl characteristics of flow fields , 2019, Pattern Recognit..

[24]  Qiong Wang,et al.  Overview of deep-learning based methods for salient object detection in videos , 2020, Pattern Recognit..

[25]  J Jos Elfring,et al.  Predicting the intention of cyclists , 2017 .

[26]  Zhihan Lv,et al.  Cyclist Social Force Model at Unsignalized Intersections With Heterogeneous Traffic , 2017, IEEE Transactions on Industrial Informatics.

[27]  Wangsheng Yu,et al.  Robust occlusion-aware part-based visual tracking with object scale adaptation , 2018, Pattern Recognit..