Noticing Motion Patterns: A Temporal CNN With a Novel Convolution Operator for Human Trajectory Prediction

As more and more robots are envisioned to cooperate with humans sharing the same space, it is desired for robots to be able to predict others’ trajectories to navigate in a safe and self-explanatory way. In this letter, we propose a Convolutional Neural Network-based approach to learn, detect, and extract patterns in sequential trajectory data, known here as Social Pattern Extraction Convolution (Social-PEC). A set of experiments carried out on the human trajectory prediction problem shows that our model performs comparably to the state of the art and outperforms in some cases. More importantly, the proposed approach unveils the obscurity in the previous use of a pooling layer, presenting a way to intuitively explain the decision-making process.

[1]  Jean Oh,et al.  Following Social Groups: Socially Compliant Autonomous Navigation in Dense Crowds , 2019, ArXiv.

[2]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[3]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[4]  Jean Oh,et al.  Social Attention: Modeling Attention in Human Crowds , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[5]  P. Molnár.  Social Force Model for Pedestrian Dynamics Typeset Using Revt E X 1 , 1995 .

[6]  Martial Hebert,et al.  Activity Forecasting , 2012, ECCV.

[7]  Andreas Krause,et al.  Unfreezing the robot: Navigation in dense, interacting crowds , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Ping Guo,et al.  Stochastic trajectory prediction with social graph network , 2019, ArXiv.

[9]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[10]  Silvio Savarese,et al.  Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks , 2019, NeurIPS.

[11]  Luc Van Gool,et al.  You'll never walk alone: Modeling social behavior for multi-target tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[12]  Dariu M. Gavrila,et al.  Human motion trajectory prediction: a survey , 2019, Int. J. Robotics Res..

[13]  Vladlen Koltun,et al.  An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling , 2018, ArXiv.

[14]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Silvio Savarese,et al.  Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[17]  Dani Lischinski,et al.  Crowds by Example , 2007, Comput. Graph. Forum.

[18]  Andreas Krause,et al.  Robot navigation in dense human crowds: the case for cooperation , 2013, 2013 IEEE International Conference on Robotics and Automation.

[19]  Kamaledin Ghiasi-Shirazi,et al.  Generalizing the Convolution Operator in Convolutional Neural Networks , 2017, Neural Processing Letters.

[20]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[21]  Abduallah A. Mohamed,et al.  Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Stefan Becker,et al.  An Evaluation of Trajectory Prediction Approaches and Notes on the TrajNet Benchmark , 2018, ArXiv.