Virtual Crowds: Steps Toward Behavioral Realism

This volume presents novel computational models for representing digital humans and their interactions with other virtual characters and meaningful environments. In this context, we describe efficient algorithms to animate, control, and author human-like agents having their own set of unique capabilities, personalities, and desires. We begin with the lowest level of footstep determination to steer agents in collision-free paths. Steering choices are controlled by navigation in complex environments, including multi-domain planning with dynamically changing situations. Virtual agents are given perceptual capabilities analogous to those of real people, including sound perception, multi-sense attention, and understanding of environment semantics which affect their behavior choices. The roles and impacts of individual attributes, such as memory and personality are explored. The animation challenges of integrating a number of simultaneous behavior and movement demands on an agent are addressed through an open source software system. Finally, the creation of stories and narratives with groups of agents subject to planning and environmental constraints culminates the presentation.

[1]  Craig W. Reynolds Flocks, herds, and schools: a distributed behavioral model , 1998 .

[2]  Vadim Bulitko,et al.  Interactive Narrative: An Intelligent Systems Approach , 2012, AI Mag..

[3]  Mei Yii Lim,et al.  Memory Models for Intelligent Social Companions , 2012, Human-Computer Interaction: The Agency Perspective.

[4]  P. Torrens,et al.  Building Agent‐Based Walking Models by Machine‐Learning on Diverse Databases of Space‐Time Trajectory Samples , 2011 .

[5]  Petros Faloutsos,et al.  Interactive motion correction and object manipulation , 2007, SIGGRAPH '08.

[6]  Dinesh K. Pai,et al.  Eyecatch: simulating visuomotor coordination for object interception , 2012, ACM Trans. Graph..

[7]  Christopher E. Peters,et al.  Fundamentals of Agent Perception and Attention Modelling , 2011 .

[8]  Mark H. Overmars,et al.  A Predictive Collision Avoidance Model for Pedestrian Simulation , 2009, MIG.

[9]  Glenn Reinman,et al.  SteerBench: a benchmark suite for evaluating steering behaviors , 2009, Comput. Animat. Virtual Worlds.

[10]  Lucas Kovar,et al.  Fast and accurate goal-directed motion synthesis for crowds , 2005, SCA '05.

[11]  Ming C. Lin,et al.  Motion planning and autonomy for virtual humans , 2008, SIGGRAPH '08.

[12]  Glenn Reinman,et al.  Scenario space: characterizing coverage, quality, and failure of steering algorithms , 2011, SCA '11.

[13]  Sébastien Paris,et al.  Pedestrian Reactive Navigation for Crowd Simulation: a Predictive Approach , 2007, Comput. Graph. Forum.

[14]  Sebastian Thrun,et al.  Anytime Dynamic A*: An Anytime, Replanning Algorithm , 2005, ICAPS.

[15]  Norman I. Badler,et al.  PAStE: A Platform for Adaptive Storytelling with Events , 2013, Intelligent Narrative Technologies.

[16]  Daniel P. W. Ellis,et al.  Ground-truth transcriptions of real music from force-aligned MIDI syntheses , 2003, ISMIR.

[17]  Glenn Reinman,et al.  Parallelized egocentric fields for autonomous navigation , 2012, The Visual Computer.

[18]  Sebastian Thrun,et al.  ARA*: Anytime A* with Provable Bounds on Sub-Optimality , 2003, NIPS.

[19]  Carol O'Sullivan,et al.  Clone attack! Perception of crowd variety , 2008, SIGGRAPH 2008.

[20]  Christopher E. Peters Direction of Attention Perception for Conversation Initiation in Virtual Environments , 2005, IVA.

[21]  Petros Faloutsos,et al.  Egocentric affordance fields in pedestrian steering , 2009, I3D '09.

[22]  Ravish Mehra,et al.  Precomputed wave simulation for real-time sound propagation of dynamic sources in complex scenes , 2010, SIGGRAPH 2010.

[23]  Sergey Levine,et al.  Space-time planning with parameterized locomotion controllers , 2011, TOGS.

[24]  Norman I. Badler,et al.  The effect of posture and dynamics on the perception of emotion , 2013, SAP.

[25]  Demetri Terzopoulos,et al.  A decision network framework for the behavioral animation of virtual humans , 2007, SCA '07.

[26]  Craig W. Reynolds Steering Behaviors For Autonomous Characters , 1999 .

[27]  Jehee Lee,et al.  Tiling Motion Patches , 2013, IEEE Trans. Vis. Comput. Graph..

[28]  John E. Laird,et al.  Enhancing intelligent agents with episodic memory , 2012, Cognitive Systems Research.

[29]  Nancy M. Amato,et al.  Roadmap-Based Level Clearing of Buildings , 2011, MIG.

[30]  David C. Rubin,et al.  Autobiographical Memory , 2019, Encyclopedia of Autism Spectrum Disorders.

[31]  Marc Cavazza,et al.  Making sense of virtual environments: action representation, grounding and common sense , 2007, IUI '07.

[32]  Michiel van de Panne,et al.  Footprint-based Quadruped Motion Synthesis , 1998, Graphics Interface.

[33]  Norman I. Badler,et al.  Virtual Crowds: Methods, Simulation, and Control , 2008, Virtual Crowds: Methods, Simulation, and Control.

[34]  Dinesh Manocha,et al.  Real-time navigation of independent agents using adaptive roadmaps , 2008, SIGGRAPH '08.

[35]  Daniel Thalmann,et al.  Navigation for digital actors based on synthetic vision, memory, and learning , 1995, Comput. Graph..

[36]  Arthur D Kuo,et al.  The six determinants of gait and the inverted pendulum analogy: A dynamic walking perspective. , 2007, Human movement science.

[37]  W. Lewis Johnson,et al.  Animated Agents for Procedural Training in Virtual Reality: Perception, Cognition, and Motor Control , 1999, Appl. Artif. Intell..

[38]  Stéphane Donikian,et al.  A synthetic-vision based steering approach for crowd simulation , 2010, ACM Transactions on Graphics.

[39]  Roland Geraerts,et al.  A navigation mesh for dynamic environments , 2012, Comput. Animat. Virtual Worlds.

[40]  Bernard Moulin,et al.  The Social Dimension of Interactions in Multiagent Systems , 1997, Agents and Multi-Agent Systems Formalisms, Methodologies, and Applications.

[41]  Petros Faloutsos,et al.  Situation agents: agent‐based externalized steering logic , 2010, Comput. Animat. Virtual Worlds.

[42]  Glenn Reinman,et al.  Improved Benchmarking for Steering Algorithms , 2011, MIG.

[43]  Glenn Reinman,et al.  Footstep navigation for dynamic crowds , 2011, SI3D.

[44]  Robert Michael Young,et al.  Managing interaction between users and agents in a multi-agent storytelling environment , 2003, AAMAS '03.

[45]  Joanna McGrenere,et al.  Affordances: Clarifying and Evolving a Concep , 2000, Graphics Interface.

[46]  Norman I. Badler,et al.  Controlling individual agents in high-density crowd simulation , 2007, SCA '07.

[47]  Tapio Lokki,et al.  Creating Interactive Virtual Acoustic Environments , 1999 .

[48]  Stéphane Donikian,et al.  Crowd of Virtual Humans: a New Approach for Real Time Navigation in Complex and Structured Environments , 2004, Comput. Graph. Forum.

[49]  N. Badler,et al.  Straight Line Walking Animation Based on Kinematic Generalization that Preserves the Original Characteristics , 1992 .

[50]  Ming C. Lin,et al.  Towards a Digital Makkah – Using Immersive 3 D Environments to Train and Prepare Pilgrims , 2011 .

[51]  Michiel van de Panne,et al.  From Footprints to Animation , 1997, Comput. Graph. Forum.

[52]  Jeff Orkin,et al.  Applying Goal-Oriented Action Planning to Games , 2008 .

[53]  Norman I. Badler,et al.  Planning approaches to constraint‐aware navigation in dynamic environments , 2015, Comput. Animat. Virtual Worlds.

[54]  Manfred Lau,et al.  Behavior planning for character animation , 2005, SCA '05.

[55]  Nathan R. Sturtevant,et al.  A Comparison of High-Level Approaches for Speeding Up Pathfinding , 2010, AIIDE.

[56]  Glenn Reinman,et al.  An Open Framework for Developing, Evaluating, and Sharing Steering Algorithms , 2009, MIG.

[57]  Earl D. Sacerdoti,et al.  The Nonlinear Nature of Plans , 1975, IJCAI.

[58]  Alberto Lacaze Hierarchical planning algorithms , 2002, SPIE Defense + Commercial Sensing.

[59]  Stefan Kopp,et al.  The Behavior Markup Language: Recent Developments and Challenges , 2007, IVA.

[60]  Jur P. van den Berg,et al.  Anytime path planning and replanning in dynamic environments , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[61]  A. Maslow A Theory of Human Motivation , 1943 .

[62]  Aaron Bryan Loyall,et al.  Believable agents: building interactive personalities , 1997 .

[63]  Jessica K. Hodgins,et al.  Generating and ranking diverse multi-character interactions , 2014, ACM Trans. Graph..

[64]  N. Badler,et al.  Crowd simulation incorporating agent psychological models, roles and communication , 2005 .

[65]  Jean-Paul Laumond,et al.  A motion capture-based control-space approach for walking mannequins: Research Articles , 2006 .

[66]  Norman I. Badler,et al.  ADAPT: The Agent Developmentand Prototyping Testbed , 2013, IEEE Transactions on Visualization and Computer Graphics.

[67]  Norman I. Badler,et al.  Animating synthetic dyadic conversations with variations based on context and agent attributes , 2012, Comput. Animat. Virtual Worlds.

[68]  Taesoo Kwon,et al.  Interactive manipulation of large-scale crowd animation , 2014, ACM Trans. Graph..

[69]  Daniel Thalmann,et al.  A Sound Propagation Model for Interagents Communication , 2000, Virtual Worlds.

[70]  Glenn Reinman,et al.  A Behavior-Authoring Framework for Multiactor Simulations , 2011, IEEE Computer Graphics and Applications.

[71]  Frank Dignum,et al.  Scalable Perception for BDI-Agents Embodied in Virtual Environments , 2011, 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology.

[72]  Murray H. Loew,et al.  Adjacency detection using quadcodes , 1987, CACM.

[73]  G. A. Miller THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[74]  Norman I. Badler,et al.  Sound localization and multi-modal steering for autonomous virtual agents , 2014, I3D.

[75]  Roland Geraerts,et al.  Real‐time density‐based crowd simulation , 2012, Comput. Animat. Virtual Worlds.

[76]  Carol O'Sullivan,et al.  Eye-catching crowds: saliency based selective variation , 2009, SIGGRAPH 2009.

[77]  David Thue,et al.  Interactive Storytelling: A Player Modelling Approach , 2007, AIIDE.

[78]  Arjan Egges,et al.  A hybrid interpolation scheme for footprint-driven walking synthesis , 2011, Graphics Interface.

[79]  John E. Laird,et al.  Using a Computer Game to Develop Advanced AI , 2001, Computer.

[80]  Rahul Narain,et al.  Aggregate dynamics for dense crowd simulation , 2009, SIGGRAPH 2009.

[81]  Neil A. Dodgson,et al.  Psychologically-based vision and attention for the simulation of human behaviour , 2005, GRAPHITE.

[82]  Steven M. LaValle,et al.  Planning algorithms , 2006 .

[83]  Carol O'Sullivan,et al.  Synthetic Vision and Memory for Autonomous Virtual Humans , 2002, Comput. Graph. Forum.

[84]  Norman I. Badler,et al.  Parameterizing Behavior Trees , 2011, MIG.

[85]  Dinesh Manocha,et al.  Reciprocal Velocity Obstacles for real-time multi-agent navigation , 2008, 2008 IEEE International Conference on Robotics and Automation.

[86]  Ken Perlin,et al.  Improv: a system for scripting interactive actors in virtual worlds , 1996, SIGGRAPH.

[87]  Yuyu Xu,et al.  Perception Markup Language: Towards a Standardized Representation of Perceived Nonverbal Behaviors , 2012, IVA.

[88]  Daniel L. Schacter,et al.  The Seven Sins of Memory: How the Mind Forgets and Remembers , 2001 .

[89]  Craig A. Knoblock,et al.  PDDL-the planning domain definition language , 1998 .

[90]  Darryl Charles,et al.  Adaptive Storytelling and Story Repair in a Dynamic Environment , 2011, ICIDS.

[91]  Rym Zalila-Wenkstern,et al.  Context-Aware Virtual Agents in Open Environments , 2010, 2010 Sixth International Conference on Autonomic and Autonomous Systems.

[92]  Nuria Pelechano,et al.  Automatic Generation of Suboptimal NavMeshes , 2011, MIG.

[93]  Olga De Troyer,et al.  An Ontology-Driven Approach for Modeling Behavior in Virtual Environments , 2005, OTM Workshops.

[94]  Norman I. Badler,et al.  Human Model Reaching, Grasping, Looking and Sitting Using Smart Objects , 2011 .

[95]  Norman I. Badler,et al.  Event-Centric Control for Background Agents , 2011, ICIDS.

[96]  Stéphane Donikian,et al.  Experiment-based modeling, simulation and validation of interactions between virtual walkers , 2009, SCA '09.

[97]  N. Badler,et al.  7-2014 ADAPT : The Agent Development and Prototyping Testbed , 2016 .

[98]  Dimitris N. Metaxas,et al.  Eurographics/ Acm Siggraph Symposium on Computer Animation (2007) Group Behavior from Video: a Data-driven Approach to Crowd Simulation , 2022 .

[99]  Jehee Lee,et al.  Synchronized multi-character motion editing , 2009, ACM Trans. Graph..

[100]  Randall W. Hill,et al.  Modeling Dynamic Perceptual Attention in Complex Virtual Environments , 2005, IVA.

[101]  Jehee Lee,et al.  Motion patches: building blocks for virtual environments annotated with motion data , 2006, ACM Trans. Graph..

[102]  Norman I. Badler,et al.  Perceptually Realistic Behavior through Alibi Generation , 2010, AIIDE.

[103]  Alejandro Beacco,et al.  Avatar Locomotion in Crowd Simulation , 2011, Int. J. Virtual Real..

[104]  S. Sternberg High-Speed Scanning in Human Memory , 1966, Science.

[105]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[106]  Thierry Siméon,et al.  Eurographics/siggraph Symposium on Computer Animation (2003) Visual Simulation of Ice Crystal Growth , 2022 .

[107]  Jan M. Allbeck,et al.  Populations with Purpose , 2011, MIG.

[108]  Alex J. Champandard,et al.  DHPA* and SHPA*: Efficient Hierarchical Pathfinding in Dynamic and Static Game Worlds , 2010, AIIDE.

[109]  Fabrice Lamarche,et al.  TopoPlan: a topological path planner for real time human navigation under floor and ceiling constraints , 2009, Comput. Graph. Forum.

[110]  Changsheng Xu,et al.  Automatic music classification and summarization , 2005, IEEE Transactions on Speech and Audio Processing.

[111]  S. Palmer,et al.  A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization. , 2012, Psychological bulletin.

[112]  Carol O'Sullivan,et al.  Metropolis: Multisensory Simulation of a Populated City , 2011, 2011 Third International Conference on Games and Virtual Worlds for Serious Applications.

[113]  Brian Magerko,et al.  AI Characters and Directors for Interactive Computer Games , 2004, AAAI.

[114]  Boyang Li,et al.  Creating Customized Game Experiences by Leveraging Human Creative Effort: A Planning Approach , 2010, AGS.

[115]  Ron Sun,et al.  Cognition and Multi-Agent Interaction: The CLARION Cognitive Architecture: Extending Cognitive Modeling to Social Simulation , 2005 .

[116]  Tsai-Yen Li,et al.  Space‐time planning in changing environments : using dynamic objects for accessibility , 2012, Comput. Animat. Virtual Worlds.

[117]  Jessica K. Hodgins,et al.  Construction and optimal search of interpolated motion graphs , 2007, ACM Trans. Graph..

[118]  Jaime G. Carbonell,et al.  Interactive drama, art and artificial intelligence , 2002 .

[119]  Henry G. Small,et al.  Co-citation in the scientific literature: A new measure of the relationship between two documents , 1973, J. Am. Soc. Inf. Sci..

[120]  Kevin O'Brien,et al.  Human Behavior Models for Agents in Simulators and Games: Part I: Enabling Science with PMFserv , 2006, Presence: Teleoperators & Virtual Environments.

[121]  Glenn Reinman,et al.  Watch Out! A Framework for Evaluating Steering Behaviors , 2008, MIG.

[122]  Norman I. Badler,et al.  Smart Events and Primed Agents , 2010, IVA.

[123]  Jean-Claude Latombe,et al.  Fast synthetic vision, memory, and learning models for virtual humans , 1999, Proceedings Computer Animation 1999.

[124]  Ari Shapiro,et al.  Building a Character Animation System , 2011, MIG.

[125]  Dani Lischinski,et al.  Crowds by Example , 2007, Comput. Graph. Forum.

[126]  Norman I. Badler,et al.  Animating human locomotion with inverse dynamics , 1996, IEEE Computer Graphics and Applications.

[127]  Taku Komura,et al.  Interaction patches for multi-character animation , 2008, ACM Trans. Graph..

[128]  Rafael Bidarra,et al.  The role of semantics in games and simulations , 2008, CIE.

[129]  E. Tulving Elements of episodic memory , 1983 .

[130]  Eric Menou Real-Time Character Animation Using Multi-layered Scripts and Spacetime Optimization , 2001, International Conference on Virtual Storytelling.

[131]  Markus Gross,et al.  Evaluating the Authoring Complexity of Interactive Narratives with Interactive Behaviour Trees , 2015 .

[132]  James J. Kuffner,et al.  Goal-Directed Navigation for Animated Characters Using Real-Time Path Planning and Control , 1998, CAPTECH.

[133]  Glenn Reinman,et al.  A modular framework for adaptive agent-based steering , 2011, SI3D.

[134]  Stéphane Donikian,et al.  A Spatial Cognitive Map and a Human-Like Memory Model Dedicated to Pedestrian Navigation in Virtual Urban Environments , 2006, Spatial Cognition.

[135]  Matthias Zwicker,et al.  Real-time planning for parameterized human motion , 2008, SCA '08.

[136]  Stacy Marsella,et al.  THESPIAN: An Architecture for Interactive Pedagogical Drama , 2005, AIED.

[137]  Steve Rabin AI Game Programming Wisdom, Vol. 2 , 2003 .

[138]  Glenn Reinman,et al.  Behavior authoring for crowd simulations , 2011, I3D '11.

[139]  Junichiro Hirayama,et al.  RoboCup Agent Learning from Observations with Hierarchical Multiple Decision Trees , 2002 .

[140]  Fabio Pittarello,et al.  Semantic description of 3D environments: a proposal based on web standards , 2006, Web3D '06.

[141]  Dinesh Manocha,et al.  Interactive navigation of multiple agents in crowded environments , 2008, I3D '08.

[142]  Céline Loscos,et al.  Intuitive crowd behavior in dense urban environments using local laws , 2003, Proceedings of Theory and Practice of Computer Graphics, 2003..

[143]  Glenn Reinman,et al.  Multi-actor Planning for Directable Simulations , 2011, 2011 Workshop on Digital Media and Digital Content Management.

[144]  Andrew Stern,et al.  Integrating Plot, Character and Natural Language Processing in the Interactive Drama Façade , 2003 .

[145]  Dirk Helbing,et al.  How simple rules determine pedestrian behavior and crowd disasters , 2011, Proceedings of the National Academy of Sciences.

[146]  Donald Lee Pieper The kinematics of manipulators under computer control , 1968 .

[147]  Jan M. Allbeck,et al.  Virtual Humans: Evolving with Common Sense , 2012, MIG.