Long-Term Pedestrian Trajectory Prediction Using Mutable Intention Filter and Warp LSTM

Trajectory prediction is one of the key capabilities for robots to safely navigate and interact with pedestrians. Critical insights from human intention and behavioral patterns need to be integrated to effectively forecast long-term pedestrian behavior. Thus, we propose a framework incorporating a mutable intention filter and a Warp LSTM (MIF-WLSTM) to simultaneously estimate human intention and perform trajectory prediction. The mutable intention filter is inspired by particle filtering and genetic algorithms, where particles represent intention hypotheses that can be mutated throughout the pedestrian’s motion. Instead of predicting sequential displacement over time, our Warp LSTM learns to generate offsets on a full trajectory predicted by a nominal intention-aware linear model, which considers the intention hypotheses during filtering process. Through experiments on a publicly available dataset, we show that our method outperforms baseline approaches and demonstrate the robust performance of our method under abnormal intention-changing scenarios.

[1]  Silvio Savarese,et al.  SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Barbara Majecka,et al.  Statistical models of pedestrian behaviour in the Forum , 2009 .

[3]  Dariu Gavrila,et al.  UvA-DARE ( Digital Academic Repository ) Pedestrian Path Prediction with Recursive Bayesian Filters : A Comparative Study , 2013 .

[4]  Siddhartha S. Srinivasa,et al.  Planning-based prediction for pedestrians , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Markus Hiller,et al.  Improvements in pedestrian movement prediction by considering multiple intentions in a Multi-Hypotheses filter , 2018, 2018 IEEE/ION Position, Location and Navigation Symposium (PLANS).

[6]  Marco Pavone,et al.  Trajectron++: Multi-Agent Generative Trajectory Forecasting With Heterogeneous Data for Control , 2020, ArXiv.

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Alberto Del Bimbo,et al.  Context-Aware Trajectory Prediction , 2017, 2018 24th International Conference on Pattern Recognition (ICPR).

[9]  Stefan Becker,et al.  RED: A Simple but Effective Baseline Predictor for the TrajNet Benchmark , 2018, ECCV Workshops.

[10]  Mark Reynolds,et al.  Bi-Prediction: Pedestrian Trajectory Prediction Based on Bidirectional LSTM Classification , 2017, 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[11]  Jodie A. Baird,et al.  Discerning intentions in dynamic human action , 2001, Trends in Cognitive Sciences.

[12]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Saeid Nahavandi,et al.  Cyclist Trajectory Prediction Using Bidirectional Recurrent Neural Networks , 2018, Australasian Conference on Artificial Intelligence.

[14]  Jean Oh,et al.  Social Attention: Modeling Attention in Human Crowds , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[15]  Gonzalo Ferrer,et al.  Behavior estimation for a complete framework for human motion prediction in crowded environments , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Emilio Frazzoli,et al.  Intention-Aware Motion Planning , 2013, WAFR.

[17]  Eike Rehder,et al.  Goal-Directed Pedestrian Prediction , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[18]  Jungwon Lee,et al.  Residual LSTM: Design of a Deep Recurrent Architecture for Distant Speech Recognition , 2017, INTERSPEECH.

[19]  Rainer Stiefelhagen,et al.  A Controlled Interactive Multiple Model Filter for Combined Pedestrian Intention Recognition and Path Prediction , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[20]  Danica Kragic,et al.  Long-term Prediction of Motion Trajectories Using Path Homology Clusters , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  R. Blake,et al.  Perception of human motion. , 2007, Annual review of psychology.

[22]  Wolfram Burgard,et al.  Feature-Based Prediction of Trajectories for Socially Compliant Navigation , 2012, Robotics: Science and Systems.

[23]  Wolfgang Hübner,et al.  Particle-based Pedestrian Path Prediction using LSTM-MDL Models , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[24]  Luc Van Gool,et al.  You'll never walk alone: Modeling social behavior for multi-target tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[25]  Rong Chen,et al.  A Theoretical Framework for Sequential Importance Sampling with Resampling , 2001, Sequential Monte Carlo Methods in Practice.

[26]  Stefano Soatto,et al.  Intent-aware long-term prediction of pedestrian motion , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[27]  Marco Pavone,et al.  Trajectron++: Dynamically-Feasible Trajectory Forecasting with Heterogeneous Data , 2020, ECCV.

[28]  Shenghua Gao,et al.  Encoding Crowd Interaction with Deep Neural Network for Pedestrian Trajectory Prediction , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[30]  Siddhartha S. Srinivasa,et al.  Legibility and predictability of robot motion , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[31]  Silvio Savarese,et al.  Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[32]  Rae-Hong Park,et al.  Residual LSTM Attention Network for Object Tracking , 2018, IEEE Signal Processing Letters.

[33]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Luis E. Ortiz,et al.  Who are you with and where are you going? , 2011, CVPR 2011.

[35]  Martial Hebert,et al.  Activity Forecasting , 2012, ECCV.

[36]  Xuejie Zhang,et al.  Using a stacked residual LSTM model for sentiment intensity prediction , 2018, Neurocomputing.

[37]  Nanning Zheng,et al.  SR-LSTM: State Refinement for LSTM Towards Pedestrian Trajectory Prediction , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Dinesh Manocha,et al.  Reciprocal n-Body Collision Avoidance , 2011, ISRR.

[39]  Dariu M. Gavrila,et al.  Human motion trajectory prediction: a survey , 2019, Int. J. Robotics Res..

[40]  Ke Xu,et al.  Online Monitoring for Safe Pedestrian-Vehicle Interactions , 2020, 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC).

[41]  Oladimeji Farri,et al.  Neural Paraphrase Generation with Stacked Residual LSTM Networks , 2016, COLING.

[42]  Fei Tian,et al.  Recurrent Residual Learning for Sequence Classification , 2016, EMNLP.

[43]  Roland Siegwart,et al.  A data-driven approach for pedestrian intention estimation , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[44]  Marco Pavone,et al.  The Trajectron: Probabilistic Multi-Agent Trajectory Modeling With Dynamic Spatiotemporal Graphs , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[45]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[46]  Gregory D. Hager,et al.  Intent-Aware Pedestrian Prediction for Adaptive Crowd Navigation , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[47]  Björn Ommer,et al.  Learning to Forecast Pedestrian Intention from Pose Dynamics , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[48]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[49]  Sridha Sridharan,et al.  Soft + Hardwired Attention: An LSTM Framework for Human Trajectory Prediction and Abnormal Event Detection , 2017, Neural Networks.

[50]  Yu Zhao,et al.  Deep Residual Bidir-LSTM for Human Activity Recognition Using Wearable Sensors , 2017, Mathematical Problems in Engineering.

[51]  John K. Tsotsos,et al.  PIE: A Large-Scale Dataset and Models for Pedestrian Intention Estimation and Trajectory Prediction , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).