Expressive Behaviors for Virtual Worlds

A person’s behavior provides significant information about their emotional state, attitudes, and attention. Our goal is to create virtual humans that convey such information to people while interacting with them in virtual worlds. The virtual humans must respond dynamically to the events surrounding them, which are fundamentally influenced by users’ actions, while providing an illusion of human-like behavior. A user must be able to interpret the dynamic cognitive and emotional state of the virtual humans using the same non-verbal cues that people use to understand one another. Toward these goals, we are integrating and extending components from three prior systems: a virtual human architecture with a wide range of cognitive and motor capabilities, a model of task-oriented emotional appraisal and socially situated planning, and a model of how emotions and coping impact physical behavior. We describe the key research issues and approach in each of these prior systems, as well as our integration and its initial implementation in a leadership training system.

[1]  Randall W. Hill,et al.  Toward the holodeck: integrating graphics, sound, character and story , 2001, AGENTS '01.

[2]  Stacy Marsella,et al.  Interactive pedagogical drama , 2000, AGENTS '00.

[3]  W. Johnson,et al.  Task-oriented collaboration with embodied agents in virtual worlds , 2001 .

[4]  C. Elliott.  The affective reasoner: a process model of emotions in a multi-agent system , 1992 .

[5]  Norman I. Badler,et al.  Dynamically altering agent behaviors using natural language instructions , 2000, AGENTS '00.

[6]  Justine Cassell,et al.  BEAT: the Behavior Expression Animation Toolkit , 2001, Life-like characters.

[7]  Shrikanth Narayanan,et al.  Limited domain synthesis of expressive military speech for animated characters , 2002, Proceedings of 2002 IEEE Workshop on Speech Synthesis, 2002..

[8]  Daniel S. Weld.  An Introduction to Least Commitment Planning , 1994, AI Mag..

[9]  W. Lewis Johnson,et al.  Extending virtual humans to support team training in virtual reality , 2003 .

[10]  S. Marsella,et al.  Interactive Pedagogical Drama for Health Interventions , 2003 .

[11]  Mark Steedman,et al.  Generating Facial Expressions for Speech , 1996, Cogn. Sci..

[12]  David R. Traum,et al.  CONVERSATION ACTS IN TASK‐ORIENTED SPOKEN DIALOGUE , 1992, Comput. Intell..

[13]  Eduard Hovy,et al.  Emotional Variation in Speech-Based Natural Language Generation , 2002 .

[14]  James C. Lester,et al.  Explanatory lifelike avatars , 1999 .

[15]  A. Newell.  Unified Theories of Cognition , 1990 .

[16]  Catherine Pelachaud,et al.  Emotional Meaning and Expression in Animated Faces , 1999, IWAI.

[17]  Stacy Marsella,et al.  Tears and fears: modeling emotions and emotional behaviors in synthetic agents , 2001, AGENTS '01.

[18]  J. Cassell,et al.  Embodied conversational agents , 2000 .

[19]  J. Cassell,et al.  Conversation as a System Framework : Designing Embodied Conversational Agents , 1999 .

[20]  S. Duncan,et al.  Some Signals and Rules for Taking Speaking Turns in Conversations , 1972 .

[21]  Subbarao Kambhampati,et al.  Planning as Refinement Search: A Unified Framework for Evaluating Design Tradeoffs in Partial-Order Planning , 1995, Artif. Intell..

[22]  David R. Traum,et al.  Negotiation over tasks in hybrid human-agent teams for simulation-based training , 2003, AAMAS '03.

[23]  Andrew Ortony,et al.  The Cognitive Structure of Emotions , 1988 .

[24]  Leonard Berkowitz,et al.  Causes and consequences of feelings , 2000 .

[25]  James C. Lester,et al.  Deictic and emotive communication in animated pedagogical agents , 2001 .

[26]  Marie desJardins,et al.  Coordinating a Distributed Planning System , 1999, AI Mag..

[27]  Mark Steedman,et al.  Animated conversation: rule-based generation of facial expression, gesture & spoken intonation for multiple conversational agents , 1994, SIGGRAPH.

[28]  Jonathan Gratch,et al.  Socially Situated Planning , 2002 .

[29]  David R Traum,et al.  Towards a Computational Theory of Grounding in Natural Language Conversation , 1991 .

[30]  Eduard Hovy,et al.  Towards Emotional Variation in Speech-Based Natural Language Processing , 2002, INLG.

[31]  Jeff Rickel,et al.  Virtual Humans for Team Training in Virtual Reality , 1999 .

[32]  Stacy Marsella,et al.  Modeling coping behavior in virtual humans: don't worry, be happy , 2003, AAMAS '03.

[33]  Justine Cassell,et al.  Human conversation as a system framework: designing embodied conversational agents , 2001 .

[34]  Kristinn R. Thórisson,et al.  The Power of a Nod and a Glance: Envelope Vs. Emotional Feedback in Animated Conversational Agents , 1999, Appl. Artif. Intell..

[35]  Catherine Pelachaud,et al.  Embodied contextual agent in information delivering application , 2002, AAMAS '02.

[36]  David R. Traum,et al.  Modelling Grounding and Discourse Obligations Using Update Rules , 2000, ANLP.

[37]  Sarit Kraus,et al.  Collaborative Plans for Complex Group Action , 1996, Artif. Intell..

[38]  Jonathan Gratch,et al.  Émile: Marshalling passions in training and education , 2000, AGENTS '00.

[39]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[40]  Sam Steel,et al.  Integrating Planning, Execution and Monitoring , 1988, AAAI.

[41]  J. Penley,et al.  Associations among the Big Five, emotional responses, and coping with acute stress , 2002 .

[42]  Norman I. Badler,et al.  Where to Look? Automating Attending Behaviors of Virtual Human Characters , 1999, Agents.

[43]  W. S. Reilly,et al.  Believable Social and Emotional Agents. , 1996 .

[44]  W. Lewis Johnson,et al.  Animated Agents for Procedural Training in Virtual Reality: Perception, Cognition, and Motor Control , 1999, Appl. Artif. Intell..

[45]  N. Freedman,et al.  Chapter 8 – The Analysis of Movement Behavior During the Clinical Interview1 , 1972 .

[46]  C. Pelachaud,et al.  Emotional meaning and expres-sion in performative faces , 1999 .

[47]  K. Scherer.  Appraisal considered as a process of multilevel sequential checking. , 2001 .

[48]  James C. Lester,et al.  Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments , 2000 .

[49]  James C. Lester,et al.  Deictic Believability: Coordinated Gesture, Locomotion, and Speech in Lifelike Pedagogical Agents , 1999, Appl. Artif. Intell..

[50]  Candace L. Sidner,et al.  Attention, Intentions, and the Structure of Discourse , 1986, CL.

[51]  R. Lazarus.  Emotion and Adaptation , 1991 .

[52]  Randall W. Hill,et al.  Toward a New Generation of Virtual Humans for Interactive Experiences , 2002, IEEE Intell. Syst..

[53]  David R. Traum,et al.  Embodied agents for multi-party dialogue in immersive virtual worlds , 2002, AAMAS '02.