Towards Equilibrium-based Interaction Modeling for Pedestrian Path Prediction

Predicting the future paths of pedestrians is shown to be crucial in various research domains, including intelligent vehicles and surveillance. Although numerous studies are conducted, the task has remained challenging, requiring a fundamental understanding of the complex human-human interactions. While current interaction-based frameworks are developed to benefit from information sharing between pedestrians, such frameworks are often ill-suited for monitoring the decisions-making process of pedestrians. In this paper, the problem of human-human interaction has been tackled by combining tools from multi-agent decision theory. Employing the game theory solution concept of Nash equilibrium, a novel equilibrium-based modification is proposed. With the aim of generalization, the proposed modification is applied to two different interaction-based frameworks and verified on publicly available datasets. Through these experiments, the advantages of the proposed modification are demonstrated.

[1]  S. Shankar Sastry,et al.  An inverse correlated equilibrium framework for utility learning in multiplayer, noncooperative settings , 2013, HiCoNS '13.

[2]  Silvio Savarese,et al.  Learning Social Etiquette: Human Trajectory Understanding In Crowded Scenes , 2016, ECCV.

[3]  Sridha Sridharan,et al.  Soft + Hardwired Attention: An LSTM Framework for Human Trajectory Prediction and Abnormal Event Detection , 2017, Neural Networks.

[4]  Julien Pettré,et al.  Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories With GANs , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[5]  Mubarak Shah,et al.  Abnormal crowd behavior detection using social force model , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Mark Reynolds,et al.  SS-LSTM: A Hierarchical LSTM Model for Pedestrian Trajectory Prediction , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[7]  M. Bierlaire,et al.  Behavioral Dynamics for Pedestrians , 2003 .

[8]  Yisong Yue,et al.  Coordinated Multi-Agent Imitation Learning , 2017, ICML.

[9]  Xiaomeng Li,et al.  Using Cellular Automata to Investigate Pedestrian Conflicts with Vehicles in Crosswalk at Signalized Intersection , 2012, Discrete Dynamics in Nature and Society.

[10]  Yanning Zhang,et al.  Human trajectory prediction in crowded scene using social-affinity Long Short-Term Memory , 2019, Pattern Recognit..

[11]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[12]  Pengyuan Shao,et al.  A more realistic simulation of pedestrian based on cellular automata , 2009, 2009 IEEE International Workshop on Open-source Software for Scientific Computation (OSSC).

[13]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[14]  Yoav Shoham,et al.  Essentials of Game Theory: A Concise Multidisciplinary Introduction , 2008, Essentials of Game Theory: A Concise Multidisciplinary Introduction.

[15]  Mohan M. Trivedi,et al.  Scene Induced Multi-Modal Trajectory Forecasting via Planning , 2019, ArXiv.

[16]  Stephen C. Adams,et al.  Multi-agent Inverse Reinforcement Learning for Certain General-sum Stochastic Games , 2019, J. Artif. Intell. Res..

[17]  Bo Zhang,et al.  Crowd Scene Understanding with Coherent Recurrent Neural Networks , 2016, IJCAI.

[18]  M. Dufwenberg.  Game theory. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[19]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[20]  Volodymyr Kuleshov,et al.  Inverse Game Theory: Learning Utilities in Succinct Games , 2015, WINE.

[21]  Sridha Sridharan,et al.  GD-GAN: Generative Adversarial Networks for Trajectory Prediction and Group Detection in Crowds , 2018, ACCV.

[22]  Stefano Ermon,et al.  Multi-Agent Generative Adversarial Imitation Learning , 2018, NeurIPS.

[23]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[24]  Gijs Dubbelman,et al.  A Survey on Path Prediction Techniques for Vulnerable Road Users: From Traditional to Deep-Learning Approaches , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[25]  Alberto Del Bimbo,et al.  Context-Aware Trajectory Prediction , 2017, 2018 24th International Conference on Pattern Recognition (ICPR).

[26]  Dirk Helbing,et al.  GENERALIZED FORCE MODEL OF TRAFFIC DYNAMICS , 1998 .

[27]  Prashant Doshi,et al.  Multi-robot inverse reinforcement learning under occlusion with interactions , 2014, AAMAS.

[28]  Jaroslaw Was,et al.  New Cellular Automata Model of Pedestrian Representation , 2006, ACRI.

[29]  Xiaogang Wang,et al.  Understanding pedestrian behaviors from stationary crowd groups , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Shenghua Gao,et al.  Encoding Crowd Interaction with Deep Neural Network for Pedestrian Trajectory Prediction , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[31]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[32]  Philip H. S. Torr,et al.  DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Lantao Yu,et al.  Multi-Agent Adversarial Inverse Reinforcement Learning , 2019, ICML.

[34]  Jun Zhu,et al.  Understanding Human Behaviors in Crowds by Imitating the Decision-Making Process , 2018, AAAI.

[35]  Silvio Savarese,et al.  SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Luis E. Ortiz,et al.  Who are you with and where are you going? , 2011, CVPR 2011.

[37]  Jie Hu,et al.  Socially-Aware Graph Convolutional Network for Human Trajectory Prediction , 2019, 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC).

[38]  Yuke Li,et al.  Which Way Are You Going? Imitative Decision Learning for Path Forecasting in Dynamic Scenes , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Martin Lauer,et al.  Pedestrian Prediction by Planning Using Deep Neural Networks , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[40]  Kevin Waugh,et al.  Computational Rationalization: The Inverse Equilibrium Problem , 2011, ICML.

[41]  Dani Lischinski,et al.  Crowds by Example , 2007, Comput. Graph. Forum.

[42]  Jean Oh,et al.  Social Attention: Modeling Attention in Human Crowds , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[43]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[44]  Katsuhiro Nishinari,et al.  Simulation for pedestrian dynamics by real-coded cellular automata (RCA) , 2007 .

[45]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Saeid Nahavandi,et al.  Long-Term Recurrent Predictive Model for Intent Prediction of Pedestrians via Inverse Reinforcement Learning , 2018, 2018 Digital Image Computing: Techniques and Applications (DICTA).

[47]  Silvio Savarese,et al.  Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[48]  Kristian Kersting,et al.  Multi-Agent Inverse Reinforcement Learning , 2010, 2010 Ninth International Conference on Machine Learning and Applications.

[49]  Luc Van Gool,et al.  You'll never walk alone: Modeling social behavior for multi-target tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[50]  Jianyi Liu,et al.  Crossing-Road Pedestrian Trajectory Prediction via Encoder-Decoder LSTM , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[51]  A. Schadschneider.  Cellular Automaton Approach to Pedestrian Dynamics - Theory , 2001, cond-mat/0112117.

[52]  Seul Jung,et al.  Pedestrian trajectory prediction via the Social‐Grid LSTM model , 2018, The Journal of Engineering.

[53]  Dongchun Ren,et al.  StarNet: Pedestrian Trajectory Prediction using Deep Neural Network in Star Topology , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[54]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[55]  Ryosuke Shibasaki,et al.  Pedestrian Trajectory Prediction in Extremely Crowded Scenarios , 2019, Sensors.

[56]  Dirk Helbing,et al.  Specification of the Social Force Pedestrian Model by Evolutionary Adjustment to Video Tracking Data , 2007, Adv. Complex Syst..

[57]  Atsushi Yamashita,et al.  Pedestrian trajectory prediction using BiRNN encoder–decoder framework* , 2019, Adv. Robotics.

[58]  I. Glicksberg.  A FURTHER GENERALIZATION OF THE KAKUTANI FIXED POINT THEOREM, WITH APPLICATION TO NASH EQUILIBRIUM POINTS , 1952 .