Design of an emotional and social interaction paradigm for the animation of 3D characters: the case of a therapy for brain injured people (the mirror neuron paradigm)

We propose an emotional and social interaction paradigm for the behavioural animation of virtual characters. Our research focuses on the production of verbal and paraverbal interactions between a human and a virtual character in the context of virtual therapy. The aim of this project is to assess virtual therapies for the rehabilitation of people with brain injuries caused by cerebral vascular accidents. The therapeutic paradigm is based on the neuroscience concept of “mirror neurones” which emphasise the perceptual side in the process of recovering neural damage. In our paradigm, the patient interacts with a virtual therapist (embodied autonomous agent) that guides and encourages him in order to fulfil the drills. Furthermore and following the “mirror neurone paradigm”, the patient can only see his virtual hands (which are handicapped). The virtual platform provides the patient with a vision of his hands undertaking a movement that he cannot perform. Doing so, we expect that the perception of this virtual movement may improve the recuperation (mirror neurone paradigm). In this paper, we also introduce the notion of an “intelligent emotional character” to produce a sense of social as well as emotional immersion in order to improve the interaction between the patient and the virtual therapist. We discuss the role of emotional interaction in the context of virtual social interaction and in particular its reflexive dimension. Finally, we present an architecture for the animation of virtual characters based on a multidisciplinary approach to model the emotional and social interaction.

[1]  K. Scherer,et al.  Appraisal processes in emotion: Theory, methods, research. , 2001 .

[2]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[3]  M J Griffin,et al.  Human factors consideration in clinical applications of virtual reality. , 1997, Studies in health technology and informatics.

[4]  J. Lannoy,et al.  Gestures and Speech: Psychological Investigations , 1991 .

[5]  Bernard Rimé,et al.  Fundamentals of nonverbal behavior , 1991 .

[6]  P. Salovey,et al.  Models of emotional intelligence , 2000 .

[7]  R. Sternberg,et al.  Handbook of Intelligence , 2000 .

[8]  Jonathan Steuer,et al.  Defining virtual reality: dimensions determining telepresence , 1992 .

[9]  A. Burkhardt.  Speech Acts, Meaning and Intentions , 1990 .

[10]  Lauri Carlson.  Dialogue Games: An Approach to Discourse Analysis , 1982 .

[11]  Maureen K. Holden,et al.  Virtual Environments for Motor Rehabilitation: Review , 2005, Cyberpsychology Behav. Soc. Netw..

[12]  Dan Sperber,et al.  La pertinence : communication et cognition , 1989 .

[13]  Heidi Sveistrup,et al.  Motor rehabilitation using virtual reality , 2004, Journal of NeuroEngineering and Rehabilitation.

[14]  Matthew Lombard,et al.  At the Heart of It All: The Concept of Presence , 2006 .

[15]  Stella Ting-Toomey.  Communicating Across Cultures , 1999 .

[16]  J. Russell.  Culture and the categorization of emotions. , 1991, Psychological bulletin.

[17]  G. Rizzolatti,et al.  Action recognition in the premotor cortex. , 1996, Brain : a journal of neurology.

[18]  Shaomin Li,et al.  The Great Leap Forward: The Transition from Relation-Based Governance to Rule-Based Governance , 2004 .

[19]  Catherine Pelachaud,et al.  From Greta's mind to her face: modelling the dynamics of affective states in a conversational embodied agent , 2003, Int. J. Hum. Comput. Stud..

[20]  G. Lakoff,et al.  The Brain's concepts: the role of the Sensory-motor system in conceptual knowledge , 2005, Cognitive neuropsychology.

[21]  Giuseppe Riva,et al.  Real Presence: How Different Ontologies Generate Different Criteria for Presence, Telepresence, and Virtual Presence , 1999, Presence.

[22]  Fausto Giunchiglia,et al.  Theories and uses of context in knowledge representation and reasoning , 2003 .

[23]  E. Goffman,et al.  Les rites d'interaction , 1974 .

[24]  J. Russell.  Core affect and the psychological construction of emotion. , 2003, Psychological review.

[25]  Philip R. Cohen,et al.  Intentions in Communication. , 1992 .

[26]  Andrea Gaggioli,et al.  Presence and rehabilitation: toward second-generation virtual reality applications in neuropsychology , 2004, Journal of NeuroEngineering and Rehabilitation.

[27]  Emily A Keshner,et al.  Virtual reality and physical rehabilitation: a new toy or a new research and rehabilitation tool? , 2004, Journal of NeuroEngineering and Rehabilitation.

[28]  Philip R. Cohen,et al.  Intentions in Communication , 1991, CL.

[29]  El Jed Mehdi.  Modelling character emotion in an interactive virtual environment , 2007 .

[30]  Peter van der Straaten.  INTERACTION AFFECTING THE SENSE OF PRESENCE IN VIRTUAL REALITY , 2000 .

[31]  O. John,et al.  Handbook of personality: Theory and research, 2nd ed. , 1999 .

[32]  Giuseppe Riva,et al.  Virtual Reality as Communication Tool: A Sociocognitive Analysis , 1999, Presence.

[33]  P. Chevalier.  The influence of social identity salience on appraisal and emotion , 2004 .

[34]  J. Mazziotta,et al.  Grasping the Intentions of Others with One's Own Mirror Neuron System , 2005, PLoS biology.

[35]  Norman I. Badler,et al.  Creating Interactive Virtual Humans: Some Assembly Required , 2002, IEEE Intell. Syst..

[36]  G. Riva,et al.  Virtual Reality in Neuro-Psycho-Physiology: Cognitive, Clinical and Methodological Issues in Assessment and Treatment , 1997 .

[37]  Armin Burkhardt,et al.  Speech acts, meaning and intentions : critical approaches to the philosophy of John R. Searle , 1990 .

[38]  Harold Garfinkel,et al.  On Formal Structures of Practical Actions , 2005 .

[39]  Michael L. Geis.  Speech acts and conversational interaction: Interactional effects , 1995 .

[40]  P. Philippot,et al.  The social context of nonverbal behavior , 1999 .