Assessment of Empathy in an Affective VR Environment using EEG Signals

With the advancements in social robotics and virtual avatars, it becomes increasingly important that these agents adapt their behavior to the mood, feelings and personality of their users. One such aspect of the user is empathy. Whereas many studies measure empathy through offline measures that are collected after empathic stimulation (e.g. post-hoc questionnaires), the current study aimed to measure empathy online, using brain activity collected during the experience. Participants watched an affective 360 video of a child experiencing domestic violence in a virtual reality headset while their EEG signals were recorded. Results showed a significant attenuation of alpha, theta and delta asymmetry in the frontal and central areas of the brain. Moreover, a significant relationship between participants' empathy scores and their frontal alpha asymmetry at baseline was found. These results demonstrate specific brain activity alterations when participants are exposed to an affective virtual reality environment, with the level of empathy as a personality trait being visible in brain activity during a baseline measurement. These findings suggest the potential of EEG measurements for development of passive brain-computer interfaces that assess the user's affective responses in real-time and consequently adapt the behavior of socially intelligent agents for a personalized interaction.

[1]  Lihua Mao,et al.  Event-related theta and alpha oscillations mediate empathy for pain , 2008, Brain Research.

[2]  Maria Luisa Gorno-Tempini,et al.  Structural anatomy of empathy in neurodegenerative disease. , 2006, Brain : a journal of neurology.

[3]  Fabio Babiloni,et al.  EEG-based Approach-Withdrawal index for the pleasantness evaluation during taste experience in realistic settings , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[4]  Arthur C. Graesser,et al.  AutoTutor and affective autotutor: Learning by talking with cognitively and emotionally intelligent computers that talk back , 2012, TIIS.

[5]  F Babiloni,et al.  Passive BCI beyond the lab: current trends and future directions , 2018, Physiological measurement.

[6]  A. Gale,et al.  Extraversion–introversion, neuroticism–stability, and EEG indicators of positive and negative empathic mood , 2001 .

[7]  P. Jackson,et al.  Suppression of Sensorimotor Alpha Power Associated With Pain Expressed by an Avatar: A Preliminary EEG Study , 2018, Front. Hum. Neurosci..

[8]  H. Pashler,et al.  Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition 1 , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[9]  Pedro F. Hernández-Ramos,et al.  How do presence, flow, and character identification affect players' empathy and interest in learning from a serious computer game? , 2016, Comput. Hum. Behav..

[10]  Christian Kothe,et al.  Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general , 2011, Journal of neural engineering.

[11]  B. Berger,et al.  Characterization of Empathy Deficits following Prefrontal Brain Damage: The Role of the Right Ventromedial Prefrontal Cortex , 2003, Journal of Cognitive Neuroscience.

[12]  T. Bettina Cornwell,et al.  A Review of Methods Utilized in Measuring Affect, Feelings, and Emotion in Advertising , 1991 .

[13]  Henrik M. Peperkorn,et al.  The impact of perception and presence on emotional reactions: a review of research in virtual reality , 2015, Front. Psychol..

[14]  R. Davidson,et al.  Prefrontal Brain Asymmetry: A Biological Substrate of the Behavioral Approach and Inhibition Systems , 1997 .

[15]  R. N. Spreng,et al.  The Toronto Empathy Questionnaire: Scale Development and Initial Validation of a Factor-Analytic Solution to Multiple Empathy Measures , 2009, Journal of personality assessment.

[16]  Guoping Li,et al.  Fourth Industrial Revolution: technological drivers, impacts and coping methods , 2017, Chinese Geographical Science.

[17]  Patrice Renaud,et al.  Virtual reality applications in forensic psychiatry , 2014, VRIC.

[18]  Max M. Louwerse,et al.  Brain Activity Reflects Sense of Presence in 360° Video for Virtual Reality , 2019, ISD.

[19]  J. Decety,et al.  “Feeling” the pain of those who are different from us: Modulation of EEG in the mu/alpha range , 2010, Cognitive, affective & behavioral neuroscience.

[20]  Peter Dalgaard,et al.  R Development Core Team (2010): R: A language and environment for statistical computing , 2010 .

[21]  John J. B. Allen,et al.  Frontal EEG asymmetry as a moderator and mediator of emotion , 2004, Biological Psychology.

[22]  J. Aharon-Peretz,et al.  Two systems for empathy: a double dissociation between emotional and cognitive empathy in inferior frontal gyrus versus ventromedial prefrontal lesions. , 2009, Brain : a journal of neurology.

[23]  Heather H. Mitchell,et al.  AutoTutor: A tutor with dialogue in natural language , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[24]  P. Zimmermann,et al.  Affective Computing—A Rationale for Measuring Mood With Mouse and Keyboard , 2003, International journal of occupational safety and ergonomics : JOSE.

[25]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[26]  P. Ekman,et al.  Approach-withdrawal and cerebral asymmetry: emotional expression and brain physiology. I. , 1990, Journal of personality and social psychology.

[27]  Richard J Davidson,et al.  Empathy is associated with dynamic change in prefrontal brain electrical activity during positive emotion in children. , 2009, Child development.

[28]  Michael Inzlicht,et al.  Right frontal cortical asymmetry predicts empathic reactions: support for a link between withdrawal motivation and empathy. , 2012, Psychophysiology.

[29]  Mike Scanlon,et al.  Am I a Bunny?: The Impact of High and Low Immersion Platforms and Viewers' Perceptions of Role on Presence, Narrative Engagement, and Empathy during an Animated 360° Video , 2018, CHI.

[30]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[31]  Aidong Zhang,et al.  A Novel Semi-Supervised Deep Learning Framework for Affective State Recognition on EEG Signals , 2014, 2014 IEEE International Conference on Bioinformatics and Bioengineering.

[32]  C. Neuper,et al.  Interactive effects of age and gender on EEG power and coherence during a short-term memory task in middle-aged adults , 2016, Neurobiology of Aging.

[33]  M. Molnár,et al.  Age-dependent features of EEG-reactivity—Spectral, complexity, and network characteristics , 2010, Neuroscience Letters.

[34]  A. J. Fridlund,et al.  Pattern recognition of self-reported emotional state from multiple-site facial EMG activity during affective imagery. , 1984, Psychophysiology.

[35]  Zhihong Zeng,et al.  Audio–Visual Affective Expression Recognition Through Multistream Fused HMM , 2008, IEEE Transactions on Multimedia.

[36]  Hongying Meng,et al.  Affective State Level Recognition in Naturalistic Facial and Vocal Expressions , 2014, IEEE Transactions on Cybernetics.

[37]  George Veletsianos Cognitive and Affective Benefits of an Animated Pedagogical Agent: Considering Contextual Relevance and Aesthetics , 2007 .

[38]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[39]  Marc Cavazza,et al.  A Brain-Computer Interface to a Plan-Based Narrative , 2013, IJCAI.

[40]  Fabio Babiloni,et al.  A new regression-based method for the eye blinks artifacts correction in the EEG signal, without using any EOG channel , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[41]  Angelica M. Tinga,et al.  Non-invasive neurophysiological measures of learning: A meta-analysis , 2019, Neuroscience & Biobehavioral Reviews.

[42]  S. Preston,et al.  Empathy: Its ultimate and proximate bases. , 2001, The Behavioral and brain sciences.

[43]  Gui-Bin Bian,et al.  Removal of Artifacts from EEG Signals: A Review , 2019, Sensors.

[44]  Dana Kulic,et al.  Affective State Estimation for Human–Robot Interaction , 2007, IEEE Transactions on Robotics.

[45]  Claude Pecheux,et al.  Mood and children: Proposition of a measurement scale , 1999 .