Lessons learned in modeling schizophrenic and depressed responsive virtual humans for training

This paper describes lessons learned in developing the linguistic, cognitive, emotional, and gestural models underlying virtual human behavior in a training application designed to train civilian police officers how to recognize gestures and verbal cues indicating different forms of mental illness and how to verbally interact with the mentally ill. Schizophrenia, paranoia, and depression were all modeled for the application. For linguistics, the application has quite complex language grammars that captured a range of syntactic structures and semantic categories. For cognition, there is a great deal of augmentation to a plan-based transition network needed to model the virtual humans knowledge. For emotions and gestures, virtual human behavior is based on expert-validated mapping tables specific to each mental illness. The paper presents five areas demanding continued research to improve virtual human behavior for use in training applications

[1]  P N Kizakevich,et al.  The virtual standardized patient. Simulated patient-practitioner dialog for patient interview training. , 2000, Studies in health technology and informatics.

[2]  Alan W. Biermann,et al.  A Measure of Semantic Complexity for Natural Language Systems , 2000 .

[3]  Robert Plutchik,et al.  The circumplex as a general model of the structure of emotions and personality. , 1997 .

[4]  P N Kizakevich,et al.  Virtual medical trainer. Patient assessment and trauma care simulator. , 1998, Studies in health technology and informatics.

[5]  Kristinn R. Thórisson,et al.  The Power of a Nod and a Glance: Envelope Vs. Emotional Feedback in Animated Conversational Agents , 1999, Appl. Artif. Intell..

[6]  John Funge,et al.  Cognitive modeling: knowledge, reasoning and planning for intelligent characters , 1999, SIGGRAPH.

[7]  Andrew Ortony,et al.  The Cognitive Structure of Emotions , 1988 .

[8]  M. Knapp,et al.  Nonverbal communication in human interaction , 1972 .

[9]  Clark Elliott.  Using the Affective Reasoner to Support Social Simulations , 1993, IJCAI.

[10]  H. Berenbaum,et al.  Emotional experience and expression in schizophrenia and depression. , 1992, Journal of abnormal psychology.

[11]  J. S. Wiggins,et al.  The five-factor model of personality : theoretical perspectives , 1996 .

[12]  James A. Russell,et al.  How shall an emotion be called , 1997 .

[13]  D. Thalmann,et al.  A behavioral animation system for autonomous actors personified by emotions , 1998 .

[14]  Nicole Chovil.  Discourse‐oriented facial displays in conversation , 1991 .

[15]  R. Krause,et al.  Facial expression of schizophrenic patients and their interaction partners. , 1989, Psychiatry.

[16]  E. André,et al.  Exploiting Models of Personality and Emotions to Control the Behavior of Animated Interactive Agents , 2000 .

[17]  Mark Steedman,et al.  Animated conversation: rule-based generation of facial expression, gesture & spoken intonation for multiple conversational agents , 1994, SIGGRAPH.

[18]  Hao Yan,et al.  More than just a pretty face: affordances of embodiment , 2000, IUI '00.

[19]  Curry Guinn,et al.  Mechanisms for dynamically changing initiative in human-computer collaborative discourse , 1996, Proceedings Third Annual Symposium on Human Interaction with Complex Systems. HICS'96.

[20]  Bruce Blumberg,et al.  The Art and Science of Synthetic Character Design , 1999 .

[21]  James C. Lester,et al.  Lifelike Pedagogical Agents for Mixed-initiative Problem Solving in Constructivist Learning Environments , 2004, User Modeling and User-Adapted Interaction.

[22]  E. Ziegel,et al.  Nonresponse In Household Interview Surveys , 1998 .

[23]  Mitsuru Ishizuka,et al.  Social role awareness in animated agents , 2001, AGENTS '01.

[24]  Curry I. Guinn,et al.  AVATALK Virtual Humans for Training with Computer Generated Forces , 2000 .

[25]  James C. Lester,et al.  Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments , 2000 .

[26]  Thomas Rist,et al.  Employing AI Methods to Control the Behavior of Animated Interface Agents , 1999, Appl. Artif. Intell..

[27]  Robert C. Hubal,et al.  Interactive Training Applications using Responsive Virtual Human Technology , 2001 .

[28]  Gary Klein.  Sources of Power , 1998 .

[29]  Jonas Beskow,et al.  Developing a 3D-agent for the august dialogue system , 1999, AVSP.

[30]  Justine Cassell,et al.  Fully Embodied Conversational Avatars: Making Communicative Behaviors Autonomous , 1999, Autonomous Agents and Multi-Agent Systems.

[31]  Ronnie W. Smith,et al.  Effects of Variable Initiative on Linguistic Behavior in Human-Computer Spoken Natural Language Dialogue , 1997, Comput. Linguistics.

[32]  Daniel Thalmann,et al.  Nonverbal communication interface for collaborative virtual environments , 1999, Virtual Reality.

[33]  B. Hayes-Roth,et al.  Knowledge Systems Laboratory December 1997 Report No . KSL 97-10 Improvisational Synthetic Actors with Flexible Personalities , 1997 .

[34]  J. Russell,et al.  Concept of Emotion Viewed From a Prototype Perspective , 1984 .

[35]  Joseph Bates,et al.  Personality-rich believable agents that use language , 1997, AGENTS '97.