Animating synthetic dyadic conversations with variations based on context and agent attributes

Conversations between two people are ubiquitous in many inhabited contexts. The kinds of conversations that occur depend on several factors, including the time, the location of the participating agents, the spatial relationship between the agents, and the type of conversation in which they are engaged. The statistical distribution of dyadic conversations among a population of agents will therefore depend on these factors. In addition, the conversation types, flow, and duration will depend on agent attributes such as interpersonal relationships, emotional state, personal priorities, and socio‐cultural proxemics. We present a framework for distributing conversations among virtual embodied agents in a real‐time simulation. To avoid generating actual language dialogues, we express variations in the conversational flow by using behavior trees implementing a set of conversation archetypes. The flow of these behavior trees depends in part on the agents' attributes and progresses based on parametrically estimated transitional probabilities. With the participating agents' state, a ‘smart event’ model steers the interchange to different possible outcomes as it executes. Example behavior trees are developed for two conversation archetypes: buyer–seller negotiations and simple asking–answering; the model can be readily extended to others. Because the conversation archetype is known to participating agents, they can animate their gestures appropriate to their conversational state. The resulting animated conversations demonstrate reasonable variety and variability within the environmental context. Copyright © 2012 John Wiley & Sons, Ltd.

[1]  Justine Cassell,et al.  Levels of Detail for Crowds and Groups , 2002, Comput. Graph. Forum.

[2]  C. O'Sullivan,et al.  Seeing is believing: body motion dominates in multisensory conversations , 2010, ACM Trans. Graph..

[3]  Stacy Marsella,et al.  SmartBody: behavior realization for embodied conversational agents , 2008, AAMAS.

[4]  Marcello Kallmann,et al.  Interaction with 3‐D Objects , 2006 .

[5]  Stéphane Donikian,et al.  A programming environment for behavioural animation , 2002, Comput. Animat. Virtual Worlds.

[6]  Daniel Thalmann,et al.  Hierarchical Model for Real Time Simulation of Virtual Human Crowds , 2001, IEEE Trans. Vis. Comput. Graph..

[7]  Hans-Peter Seidel,et al.  Real-time lens blur effects and focus control , 2010, SIGGRAPH 2010.

[8]  Hannes Högni Vilhjálmsson,et al.  Social Perception and Steering for Online Avatars , 2008, IVA.

[9]  Bernard Moulin,et al.  Chapter 9. An approach for modelling and simulating conversations , 2001 .

[10]  Mitsuru Ishizuka,et al.  MPML3D: Scripting Agents for the 3D Internet , 2011, IEEE Transactions on Visualization and Computer Graphics.

[11]  Dinesh Manocha,et al.  Composite agents , 2008, SCA '08.

[12]  Catherine Pelachaud,et al.  Performative faces , 1998, Speech Commun..

[13]  Justine Cassell,et al.  BEAT: the Behavior Expression Animation Toolkit , 2001, Life-like characters.

[14]  David R. Traum,et al.  Dialog Simulation for Background Characters , 2005, IVA.

[15]  Daniel Thalmann,et al.  Handbook of Virtual Humans , 2004 .

[16]  Kristinn R. Thórisson,et al.  A Granular Architecture for Dynamic Realtime Dialogue , 2008, IVA.

[17]  Hadas Kress-Gazit,et al.  Constraints-Based Complex Behavior in Rich Environments , 2010, IVA.

[18]  Cathy Ennis Plausible crowd and group formations , 2011 .

[19]  David A. Forsyth,et al.  Generalizing motion edits with Gaussian processes , 2009, ACM Trans. Graph..

[20]  Carol O'Sullivan,et al.  Clone attack! Perception of crowd variety , 2008, ACM Trans. Graph..

[21]  Norman I. Badler,et al.  Virtual Crowds: Methods, Simulation, and Control , 2008, Virtual Crowds: Methods, Simulation, and Control.

[22]  Joseph K. Kearney,et al.  Controlling steering behavior for small groups of pedestrians in virtual urban environments , 2002 .

[23]  Petros Faloutsos,et al.  Situation agents: agent‐based externalized steering logic , 2010, Comput. Animat. Virtual Worlds.

[24]  David R. Traum,et al.  Dynamic movement and positioning of embodied agents in multiparty conversations , 2007, AAMAS '07.

[25]  Stefan Kopp,et al.  Towards a Common Framework for Multimodal Generation: The Behavior Markup Language , 2006, IVA.

[26]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[27]  David G. Novick,et al.  A Computational Model of Culture-Specific Conversational Behavior , 2007, IVA.

[28]  Stefan Kopp,et al.  The Behavior Markup Language: Recent Developments and Challenges , 2007, IVA.

[29]  Demetri Terzopoulos,et al.  Autonomous pedestrians , 2005, SCA '05.

[30]  Luc Van Gool,et al.  Populating Ancient Pompeii with Crowds of Virtual Romans , 2007, VAST.

[31]  Sergey Levine,et al.  Real-time prosody-driven synthesis of body language , 2009, ACM Trans. Graph..

[32]  Hans-Peter Seidel,et al.  Annotated New Text Engine Animation Animation Lexicon Animation Gesture Profiles MR : . . . JL : . . . Gesture Generation Video Annotated Gesture Script , 2007 .

[33]  Norman I. Badler,et al.  Smart Events and Primed Agents , 2010, IVA.

[34]  Demetri Terzopoulos,et al.  A decision network framework for the behavioral animation of virtual humans , 2007, SCA '07.