EEVEE: the Empathy-Enhancing Virtual Evolving Environment

Empathy is a multifaceted emotional and mental faculty that is often found to be affected in a great number of psychopathologies, such as schizophrenia, yet it remains very difficult to measure in an ecological context. The challenge stems partly from the complexity and fluidity of this social process, but also from its covert nature. One powerful tool to enhance experimental control over such dynamic social interactions has been the use of avatars in virtual reality (VR); information about an individual in such an interaction can be collected through the analysis of his or her neurophysiological and behavioral responses. We have developed a unique platform, the Empathy-Enhancing Virtual Evolving Environment (EEVEE), which is built around three main components: (1) different avatars capable of expressing feelings and emotions at various levels based on the Facial Action Coding System (FACS); (2) systems for measuring the physiological responses of the observer (heart and respiration rate, skin conductance, gaze and eye movements, facial expression); and (3) a multimodal interface linking the avatar's behavior to the observer's neurophysiological response. In this article, we provide a detailed description of the components of this innovative platform and validation data from the first phases of development. Our data show that healthy adults can discriminate different negative emotions, including pain, expressed by avatars at varying intensities. We also provide evidence that masking part of an avatar's face (top or bottom half) does not prevent the detection of different levels of pain. This innovative and flexible platform provides a unique tool to study and even modulate empathy in a comprehensive and ecological manner in various populations, notably individuals suffering from neurological or psychiatric disorders.

[1]  Rudolf A. Makkreel,et al.  Wilhelm Dilthey: Selected Works, Volume I: Introduction to the Human Sciences , 1991 .

[2]  Sridha Sridharan,et al.  Automatically Detecting Pain in Video Through Facial Action Units , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  P. Jackson,et al.  On what ground do we mentalize? Characteristics of current tasks and sources of information that contribute to mentalizing judgments. , 2013, Psychological assessment.

[4]  Jonathan Gratch,et al.  Humans versus Computers: Impact of Emotion Expressions on People's Decision Making , 2015, IEEE Transactions on Affective Computing.

[5]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[6]  H. Bülthoff Dynamic Facial Expressions , 2001 .

[7]  Philip L. Jackson,et al.  The modulation of somatosensory resonance by psychopathic traits and empathy , 2013, Front. Hum. Neurosci..

[8]  Jean Decety,et al.  Empathy and judging other's pain: an fMRI study of alexithymia. , 2007, Cerebral cortex.

[9]  Qianfeng Wang,et al.  Exposure to violence reduces empathetic responses to other’s pain , 2013, Brain and Cognition.

[10]  K. Prkachin Assessing pain by facial expression: facial expression as nexus. , 2009, Pain research & management.

[11]  Yong Tao,et al.  Compound facial expressions of emotion , 2014, Proceedings of the National Academy of Sciences.

[12]  K. Prkachin,et al.  The structure, reliability and validity of pain expression: Evidence from patients with shoulder pain , 2008, PAIN.

[13]  K. Prkachin The consistency of facial expressions of pain: a comparison across modalities , 1992, Pain.

[14]  Pierre Rainville,et al.  Brain responses to dynamic facial expressions of pain , 2006, Pain.

[15]  Feeling but not caring: Empathic alteration in narcissistic men with high psychopathic traits , 2014, Psychiatry Research: Neuroimaging.

[16]  J. Decety,et al.  To what extent do we share the pain of others? Insight from the neural bases of pain empathy , 2006, Pain.

[17]  Klaus Mathiak,et al.  Recognition Profile of Emotions in Natural and Virtual Faces , 2008, PloS one.

[18]  Oliver G. B. Garrod,et al.  Dynamic Facial Expressions of Emotion Transmit an Evolving Hierarchy of Signals over Time , 2014, Current Biology.

[19]  S. Baron-Cohen,et al.  Measuring empathy: reliability and validity of the Empathy Quotient , 2004, Psychological Medicine.

[20]  Pedro V. Sander,et al.  Texture mapping progressive meshes , 2001, SIGGRAPH.

[21]  R. Schulz,et al.  Attachment Insecurity and Perceived Partner Suffering as Predictors of Personal Distress. , 2010, Journal of experimental social psychology.

[22]  Heloir,et al.  The Uncanny Valley , 2019, The Animation Studies Reader.

[23]  G. Rizzolatti,et al.  Both of Us Disgusted in My Insula The Common Neural Basis of Seeing and Feeling Disgust , 2003, Neuron.

[24]  J. Jennings,et al.  Spouses' cardiovascular reactivity to their partners' suffering. , 2010, The journals of gerontology. Series B, Psychological sciences and social sciences.

[25]  Eric W. Sellers,et al.  Predictive Spelling With a P300-Based Brain–Computer Interface: Increasing the Rate of Communication , 2010, Int. J. Hum. Comput. Interact..

[26]  T. Whalen,et al.  Recognizing Emotions on Static and Animated Avatar Faces , 2006, 2006 IEEE International Workshop on Haptic Audio Visual Environments and their Applications (HAVE 2006).

[27]  Jean Decety,et al.  Expertise Modulates the Perception of Pain in Others , 2007, Current Biology.

[28]  J. Decety,et al.  Empathy in Mental Illness: The cognitive neuropsychology of empathy , 2007 .

[29]  Amanda C.C. Williams,et al.  Pain and negative emotions in the face: judgements by health care professionals , 2002, PAIN.

[30]  Athanasios V. Vasilakos,et al.  Guest editorial: special section on affective and pervasive computing for healthcare , 2010, IEEE Trans. Inf. Technol. Biomed..

[31]  Jun'ichiro Seyama,et al.  The Uncanny Valley: Effect of Realism on the Impression of Artificial Human Faces , 2007, PRESENCE: Teleoperators and Virtual Environments.

[32]  Pierre Rainville,et al.  Are both the sensory and the affective dimensions of pain encoded in the face? , 2012, PAIN.

[33]  Jeffrey F. Cohn,et al.  Painful data: The UNBC-McMaster shoulder pain expression archive database , 2011, Face and Gesture 2011.

[34]  M. Robinson,et al.  Gender bias in the observation of experimental pain , 2003, Pain.

[35]  Claus Lamm,et al.  Meta-analytic evidence for common and distinct neural networks associated with directly experienced pain and empathy for pain , 2011, NeuroImage.

[36]  Andrew N. Meltzoff,et al.  How do we perceive the pain of others? A window into the neural processes involved in empathy , 2005, NeuroImage.

[37]  Eva G Krumhuber,et al.  FACSGen 2.0 animation software: generating three-dimensional FACS-valid facial expressions for emotion research. , 2012, Emotion.

[38]  Milind Tambe,et al.  A Study of Emotional Contagion with Virtual Characters , 2012, IVA.

[39]  Stephanie Lay,et al.  The Uncanny Valley Effect , 2015 .

[40]  G. Hein,et al.  Skin Conductance Response to the Pain of Others Predicts Later Costly Helping , 2011, PloS one.

[41]  K. Craig,et al.  A biopsychosocial formulation of pain communication. , 2011, Psychological bulletin.

[42]  J. Decety,et al.  The functional architecture of human empathy. , 2004, Behavioral and cognitive neuroscience reviews.

[43]  K. Vogeley,et al.  Toward a second-person neuroscience 1 , 2013, Behavioral and Brain Sciences.

[44]  Sandra E. Ward,et al.  Reducing racial disparities in pain treatment: The role of empathy and perspective-taking , 2011, PAIN.

[45]  R. Descartes,et al.  The passions of the soul , 1989 .

[46]  Pascal Belin,et al.  Recognition and discrimination of prototypical dynamic expressions of pain and emotions , 2008, PAIN®.

[47]  K. Ochsner,et al.  The neuroscience of empathy: progress, pitfalls and promise , 2012, Nature Neuroscience.

[48]  Jean Decety,et al.  Physicians down-regulate their pain empathy response: An event-related brain potential study , 2010, NeuroImage.

[49]  Amy P Abernethy,et al.  Enhancing Communication Between Oncologists and Patients With a Computer-Based Training Program , 2011, Annals of Internal Medicine.

[50]  P. Jackson,et al.  Stimulating the brain to study social interactions and empathy , 2012, Brain Stimulation.

[51]  Louette R. Johnson Lutjens Research , 2006 .

[52]  Mehdi Ammi,et al.  The Impact of Combining Kinesthetic and Facial Expression Displays on Emotion Recognition by Users , 2014, Int. J. Hum. Comput. Interact..

[53]  Daniel Fiset,et al.  A refined examination of the facial cues contributing to vicarious effects on self-pain and spinal responses. , 2013, The journal of pain : official journal of the American Pain Society.

[54]  Fanny Eugène,et al.  Perception of pain in others: implication for caregivers. , 2011, Pain management.

[55]  Felipe Fregni,et al.  Modulation of emotions associated with images of human pain using anodal transcranial direct current stimulation (tDCS) , 2009, Neuropsychologia.

[56]  Klaus R. Scherer,et al.  The perception of changing emotion expressions , 2012, Cognition & emotion.