Using computer vision to simulate the motion of virtual agents

In this paper, we propose a new model to simulate the movement of virtual humans based on trajectories captured automatically from filmed video sequences. These trajectories are grouped into similar classes using an unsupervised clustering algorithm, and an extrapolated velocity field is generated for each class. A physically‐based simulator is then used to animate virtual humans, aiming to reproduce the trajectories fed to the algorithm and at the same time avoiding collisions with other agents. The proposed approach provides an automatic way to reproduce the motion of real people in a virtual environment, allowing the user to change the number of simulated agents while keeping the same goals observed in the filmed video. Copyright © 2007 John Wiley & Sons, Ltd.

[1]  Craig W. Reynolds.  Flocks, herds, and schools: a distributed behavioral model , 1987, SIGGRAPH.

[2]  John C. Davis,et al.  Contouring: A Guide to the Analysis and Display of Spatial Data , 1992 .

[3]  Demetri Terzopoulos,et al.  Artificial fishes: physics, locomotion, perception, behavior , 1994, SIGGRAPH.

[4]  Eric Bouvier,et al.  From crowd simulation to airbag deployment: particle systems, a new paradigm of simulation , 1997, J. Electronic Imaging.

[5]  Jerry L. Prince,et al.  Snakes, shapes, and gradient vector flow , 1998, IEEE Trans. Image Process..

[6]  D. Helbing,et al.  Self-Organization Phenomena in Pedestrian Crowds , 1998, cond-mat/9806152.

[7]  Siome Goldenstein,et al.  Non-linear dynamical system approach to behavior modeling , 1999, The Visual Computer.

[8]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Dirk Helbing,et al.  Simulating dynamical features of escape panic , 2000, Nature.

[10]  Daniel Thalmann,et al.  Crowd simulation for interactive virtual environments and VRtraining systems , 2001 .

[11]  Daniel Thalmann,et al.  Hierarchical Model for Real Time Simulation of Virtual Human Crowds , 2001, IEEE Trans. Vis. Comput. Graph..

[12]  L. Davis,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002, Proc. IEEE.

[13]  Anil K. Jain,et al.  Unsupervised Learning of Finite Mixture Models , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Soraia Raupp Musse,et al.  Hierarchical model for real time simulation of virtual human crowds , 2001, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[15]  I. Haritaoglu,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002 .

[16]  Dirk Helbing,et al.  The social force pedestrian model applied to real life scenarios , 2003 .

[17]  David C. Brogan,et al.  Realistic human walking paths , 2003, Proceedings 11th IEEE International Workshop on Program Comprehension.

[18]  Rita Cucchiara,et al.  Detecting Moving Objects, Ghosts, and Shadows in Video Streams , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Tieniu Tan,et al.  People tracking based on motion model and motion constraints with automatic initialization , 2004, Pattern Recognit..

[20]  Mubarak Shah,et al.  Multi feature path modeling for video surveillance , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[21]  Jessica K. Hodgins,et al.  Reactive pedestrian path following from examples , 2004, The Visual Computer.

[22]  IMAG-LIFIA,et al.  Comparison of Correlation Techniques , 2004 .

[23]  Stephen Chenney,et al.  Flow tiles , 2004, SCA '04.

[24]  Daniel Thalmann,et al.  Crowdbrush: interactive authoring of real-time crowd scenes , 2004, SCA '04.

[25]  Tim J. Ellis,et al.  Learning semantic scene models from observing activity in visual surveillance , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  Yu-Chi Lai,et al.  Group motion graphs , 2005, SCA '05.

[27]  Max Lu,et al.  Robust and efficient foreground analysis for real-time video surveillance , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[28]  Lubos Buzna,et al.  Self-Organized Pedestrian Crowd Dynamics: Experiments, Simulations, and Design Solutions , 2005, Transp. Sci..

[29]  Soraia Raupp Musse,et al.  A Background Subtraction Model Adapted to Illumination Changes , 2006, 2006 International Conference on Image Processing.

[30]  Adrien Treuille,et al.  Continuum crowds , 2006, SIGGRAPH 2006.

[31]  Jean-Paul Laumond,et al.  Real-time navigating crowds: scalable simulation and rendering: Research Articles , 2006 .

[32]  Norman I. Badler,et al.  Modeling realistic high density autonomous agent crowd movement: social forces, communication, roles and psychological influences , 2006 .

[33]  Daniel Thalmann,et al.  Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/cav.147 , 2022 .

[34]  Fang-Hsuan Cheng,et al.  Real time multiple objects tracking and identification based on discrete wavelet transform , 2006, Pattern Recognit..