Coupling camera-tracked humans with a simulated virtual crowd

Our objective with this paper is to show how we can couple a group of real people and a simulated crowd of virtual humans. We attach group behaviors to the simulated humans to get a plausible reaction to real people. We use a two stage system: in the first stage, a group of people are segmented from a live video, then a human detector algorithm extracts the positions of the people in the video, which are finally used to feed the second stage, the simulation system. The positions obtained by this process allow the second module to render the real humans as avatars in the scene, while the behavior of additional virtual humans is determined by using a simulation based on a social forces model. Developing the method required three specific contributions: a GPU implementation of the codebook algorithm that includes an auxiliary codebook to improve the background subtraction against illumination changes; the use of semantic local binary patterns as a human descriptor; the parallelization of a social forces model, in which we solve a case of agents merging with each other. The experimental results show how a large virtual crowd reacts to over a dozen humans in a real environment.

[1]  Larry S. Davis,et al.  Real-time foreground-background segmentation using codebook model , 2005, Real Time Imaging.

[2]  Craig W. Reynolds.  Flocks, herds, and schools: a distributed behavioral model , 1998 .

[3]  Paolo Fiorini,et al.  Motion Planning in Dynamic Environments Using Velocity Obstacles , 1998, Int. J. Robotics Res..

[4]  Fatih Murat Porikli,et al.  Human Detection via Classification on Riemannian Manifolds , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  OjalaTimo,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002 .

[6]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Tieniu Tan,et al.  Rapid and robust human detection and tracking based on omega-shape features , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[8]  Libo Sun,et al.  A Data-Driven Approach for Simulating Pedestrian Collision Avoidance in Crossroads , 2011, 2011 Workshop on Digital Media and Digital Content Management.

[9]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Dinesh Manocha,et al.  Reciprocal Velocity Obstacles for real-time multi-agent navigation , 2008, 2008 IEEE International Conference on Robotics and Automation.

[12]  H. Abdul Rauf,et al.  Edgelet based human detection and tracking by combined segmentation and soft decision , 2009, 2009 International Conference on Control, Automation, Communication and Energy Conservation.

[13]  Rytis Maskeliunas,et al.  Application Of Computer Vision Systems For Passenger Counting In Public Transport , 2013 .

[14]  Adrien Treuille,et al.  Continuum crowds , 2006, ACM Trans. Graph..

[15]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[16]  Norman I. Badler,et al.  Being a part of the crowd: towards validating VR crowds using presence , 2008, AAMAS.

[17]  Paul A. Viola,et al.  Detecting Pedestrians Using Patterns of Motion and Appearance , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[18]  Daniel Thalmann,et al.  An immersive multi-agent system for interactive applications , 2012, The Visual Computer.

[19]  Dimitris N. Metaxas,et al.  Eurographics/ Acm Siggraph Symposium on Computer Animation (2007) Group Behavior from Video: a Data-driven Approach to Crowd Simulation , 2022 .

[20]  Dirk Helbing,et al.  Simulating dynamical features of escape panic , 2000, Nature.

[21]  D. Helbing,et al.  The Walking Behaviour of Pedestrian Social Groups and Its Impact on Crowd Dynamics , 2010, PloS one.

[22]  Dinesh Manocha,et al.  Reciprocal n-Body Collision Avoidance , 2011, ISRR.

[23]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[24]  Qunsheng Peng,et al.  Inserting virtual pedestrians into pedestrian groups video with behavior consistency , 2013, The Visual Computer.

[25]  李蔡彥,et al.  Interactively Directing Virtual Crowds in a Virtual Environment , 2000 .

[26]  Kiyoharu Aizawa,et al.  Tracking of humans and estimation of body/head orientation from top-view single camera for visual focus of attention analysis , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[27]  Stéphane Donikian,et al.  A synthetic-vision based steering approach for crowd simulation , 2010, ACM Transactions on Graphics.

[28]  Karen Das,et al.  Omega Model for Human Detection and Counting for application in Smart Surveillance System , 2013, ArXiv.

[29]  T. Vicsek,et al.  Simulation of pedestrian crowds in normal and evacuation situations , 2002 .

[30]  Avi Bleiweiss,et al.  Multi Agent Navigation on the GPU , 2011 .

[31]  Claudio Rosito Jung,et al.  Using computer vision to simulate the motion of virtual agents: Research Articles , 2007 .

[32]  Soraia Raupp Musse,et al.  Using computer vision to simulate the motion of virtual agents , 2007, Comput. Animat. Virtual Worlds.

[33]  Dani Lischinski,et al.  Crowds by Example , 2007, Comput. Graph. Forum.

[34]  Somnath Sengupta,et al.  Human Motion Detection and Tracking for Video Surveillance , 2007 .

[35]  Xin Wang,et al.  Data-Driven Macroscopic Crowd Animation Synthesis Method using Velocity Fields , 2008, 2008 International Symposium on Computational Intelligence and Design.

[36]  Shuicheng Yan,et al.  Discriminative local binary patterns for human detection in personal album , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.