Scenario Generalization of Data-driven Imitation Models in Crowd Simulation

Crowd simulation, the study of the movement of multiple agents in complex environments, presents a unique application domain for machine learning. One challenge in crowd simulation is to imitate the movement of expert agents in highly dense crowds. An imitation model could substitute an expert agent if the model behaves as good as the expert. This will bring many exciting applications. However, we believe no prior studies have considered the critical question of how training data and training methods affect imitators when these models are applied to novel scenarios. In this work, a general imitation model is represented by applying either the Behavior Cloning (BC) training method or a more sophisticated Generative Adversarial Imitation Learning (GAIL) method, on three typical types of data domains: standard benchmarks for evaluating crowd models, random sampling of state-action pairs, and egocentric scenarios that capture local interactions. Simulated results suggest that (i) simpler training methods are overall better than more complex training methods, (ii) training samples with diverse agent-agent and agent-obstacle interactions are beneficial for reducing collisions when the trained models are applied to new scenarios. We additionally evaluated our models in their ability to imitate real world crowd trajectories observed from surveillance videos. Our findings indicate that models trained on representative scenarios generalize to new, unseen situations observed in real human crowds.

[1]  Dinesh Manocha,et al.  Velocity-based modeling of physical interactions in multi-agent simulations , 2013, SCA '13.

[2]  Jungdam Won,et al.  Crowd simulation by deep reinforcement learning , 2018, MIG.

[3]  Dinesh Manocha,et al.  Modeling, Simulation and Visual Analysis of Crowds: A Multidisciplinary Perspective , 2013, Modeling, Simulation and Visual Analysis of Crowds.

[4]  Mykel J. Kochenderfer,et al.  Imitating driver behavior with generative adversarial networks , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[5]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[6]  Glenn Reinman,et al.  SteerBench: a benchmark suite for evaluating steering behaviors , 2009, Comput. Animat. Virtual Worlds.

[7]  Soraia Raupp Musse,et al.  Optimal Group Distribution based on Thermal and Psycho-Social Aspects , 2019, CASA.

[8]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[9]  Lisa Torrey,et al.  Crowd Simulation Via Multi-Agent Reinforcement Learning , 2010, AIIDE.

[10]  Dinh Q. Phung,et al.  On Deep Domain Adaptation: Some Theoretical Understandings , 2018 .

[11]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[12]  Qunsheng Peng,et al.  Group Modeling: A Unified Velocity‐Based Approach , 2017, Comput. Graph. Forum.

[13]  Paolo Fiorini,et al.  Motion Planning in Dynamic Environments Using Velocity Obstacles , 1998, Int. J. Robotics Res..

[14]  Prashant Doshi,et al.  A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress , 2018, Artif. Intell..

[15]  Dinesh Manocha,et al.  Modeling, Simulation and Visual Analysis of Crowds , 2013, The International Series in Video Computing.

[16]  Luiselena Casadiego Bastidas Social crowd controllers using reinforcement learning methods , 2014 .

[17]  Gang Qiao,et al.  The Role of Data-driven Priors in Multi-agent Crowd Trajectory Estimation , 2017, AAAI.

[18]  Dinesh Manocha,et al.  Reciprocal n-Body Collision Avoidance , 2011, ISRR.

[19]  Xiaodong Gu,et al.  Data-Driven and Collision-Free Hybrid Crowd Simulation Model for Real Scenario , 2018, ICONIP.

[20]  Philip Chan,et al.  Toward accurate dynamic time warping in linear time and space , 2007, Intell. Data Anal..

[22]  José M. F. Moura,et al.  Adversarial Multiple Source Domain Adaptation , 2018, NeurIPS.

[23]  Vladimir Pavlovic,et al.  Characterizing the relationship between environment layout and crowd movement using machine learning , 2017, MIG.

[24]  Michael H. Bowling,et al.  Apprenticeship learning using linear programming , 2008, ICML '08.

[25]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[26]  Glenn Reinman,et al.  SteerBench: a benchmark suite for evaluating steering behaviors , 2009 .

[27]  Dinesh Manocha,et al.  Interactive simulation of dynamic crowd behaviors using general adaptation syndrome theory , 2012, I3D '12.

[28]  Vladimir Pavlovic,et al.  Filling in the blanks: reconstructing microscopic crowd motion from multiple disparate noisy sensors , 2016, 2016 IEEE Winter Applications of Computer Vision Workshops (WACVW).

[29]  John R. Hershey,et al.  Single-Channel Multitalker Speech Recognition , 2010, IEEE Signal Processing Magazine.

[30]  J. Andrew Bagnell,et al.  Efficient Reductions for Imitation Learning , 2010, AISTATS.

[31]  Glenn Reinman,et al.  Scenario space: characterizing coverage, quality, and failure of steering algorithms , 2011, SCA '11.

[32]  Jean-Baptiste Mouret,et al.  Bayesian Optimization with Automatic Prior Selection for Data-Efficient Direct Policy Search , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[33]  Stefano Ermon,et al.  Generative Adversarial Imitation Learning , 2016, NIPS.

[34]  Trung Le,et al.  Theoretical Perspective of Deep Domain Adaptation , 2018, ArXiv.

[35]  Peter Stone,et al.  Behavioral Cloning from Observation , 2018, IJCAI.

[36]  Sergey Levine,et al.  A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models , 2016, ArXiv.

[37]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[38]  Ioannis Karamouzas,et al.  Universal power law governing pedestrian interactions. , 2014, Physical review letters.

[39]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Hao Zhang,et al.  Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[41]  Norman I. Badler,et al.  Virtual Crowds: Steps Toward Behavioral Realism , 2015, Virtual Crowds: Steps Toward Behavioral Realism.

[42]  Jia Pan,et al.  Deep-Learned Collision Avoidance Policy for Distributed Multiagent Navigation , 2016, IEEE Robotics and Automation Letters.

[43]  Soraia Raupp Musse,et al.  Crowd Analysis Using Computer Vision Techniques , 2010, IEEE Signal Processing Magazine.

[44]  Vicsek,et al.  Novel type of phase transition in a system of self-driven particles. , 1995, Physical review letters.