Towards emotion recognition for virtual environments: an evaluation of eeg features on benchmark dataset

One of the challenges in virtual environments is the difficulty users have in interacting with these increasingly complex systems. Ultimately, endowing machines with the ability to perceive users emotions will enable a more intuitive and reliable interaction. Consequently, using the electroencephalogram as a bio-signal sensor, the affective state of a user can be modelled and subsequently utilised in order to achieve a system that can recognise and react to the user’s emotions. This paper investigates features extracted from electroencephalogram signals for the purpose of affective state modelling based on Russell’s Circumplex Model. Investigations are presented that aim to provide the foundation for future work in modelling user affect to enhance interaction experience in virtual environments. The DEAP dataset was used within this work, along with a Support Vector Machine and Random Forest, which yielded reasonable classification accuracies for Valence and Arousal using feature vectors based on statistical measurements and band power from the α, β, δ, and 𝜃 waves and High Order Crossing of the EEG signal.

[1]  Cristina Botella,et al.  Internet and Virtual Reality as Assessment and Rehabilitation Tools for Clinical Psychology and Neuroscience 3 Virtual Reality and Psychotherapy , 2022 .

[2]  Neera Jain,et al.  Real-Time Sensing of Trust in Human-Machine Interactions , 2016 .

[3]  Jing Fan,et al.  Multimodal adaptive social interaction in virtual environment (MASI-VR) for children with Autism spectrum disorders (ASD) , 2016, 2016 IEEE Virtual Reality (VR).

[4]  Ricardo A. Ramirez-Mendoza,et al.  Advanced driver monitoring for assistance system (ADMAS) , 2018 .

[5]  Panagiotis D. Bamidis,et al.  Real time emotion aware applications: A case study employing emotion evocative pictures and neuro-physiological sensing enhanced by Graphic Processor Units , 2012, Comput. Methods Programs Biomed..

[6]  Grigore C. Burdea,et al.  Treating Psychological and Physical Disorders with VR , 2022 .

[7]  Sebastian Bader,et al.  Controlling Smart Environments using a Brain Computer Interface , 2010 .

[8]  Joseph J. Lim,et al.  High-fidelity facial and speech animation for VR HMDs , 2016, ACM Trans. Graph..

[9]  Yuan-Pin Lin,et al.  EEG-Based Emotion Recognition in Music Listening , 2010, IEEE Transactions on Biomedical Engineering.

[10]  R. Davidson.  Anterior cerebral asymmetry and the nature of emotion , 1992, Brain and Cognition.

[11]  Shao-Wei Lu,et al.  EEG-based brain-computer interface for smart living environmental auto-adjustment , 2010 .

[12]  Jonathan R. Wolpaw.  Brain-computer interfaces: progress, problems, and possibilities , 2012, IHI '12.

[13]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[14]  G. Pfurtscheller,et al.  Presence Research and EEG , 2002 .

[15]  Bilge Mutlu,et al.  Pay attention!: designing adaptive agents that monitor and improve user engagement , 2012, CHI.

[16]  Laura Aymerich-Franch,et al.  Presence and Emotions in Playing a Group Game in a Virtual Environment: The Influence of Body Participation , 2010, Cyberpsychology Behav. Soc. Netw..

[17]  Martin Buss,et al.  Feature Extraction and Selection for Emotion Recognition from EEG , 2014, IEEE Transactions on Affective Computing.

[18]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  A. Walker.  Electroencephalography, Basic Principles, Clinical Applications and Related Fields , 1982 .

[20]  Albert Rizzo,et al.  Virtual reality for psychotherapy: Current reality and future possibilities. , 2003 .

[21]  Ann L. Brown,et al.  How people learn: Brain, mind, experience, and school. , 1999 .

[22]  Miyoung Kim,et al.  A Review on the Computational Methods for Emotional State Estimation from the Human EEG , 2013, Comput. Math. Methods Medicine.

[23]  Brendan Z. Allison,et al.  Toward Ubiquitous BCIs , 2009 .

[24]  Desney S. Tan,et al.  Brain-Computer Interfacing for Intelligent Systems , 2008, IEEE Intelligent Systems.

[25]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[26]  Rosalind W. Picard.  Affective computing: challenges , 2003, Int. J. Hum. Comput. Stud..

[27]  Yasmín Hernández,et al.  Virtual Reality and Affective Computing for Improving Learning , 2013, Res. Comput. Sci..

[28]  Mariano Alcañiz Raya,et al.  Affective Interactions Using Virtual Reality: The Link between Presence and Emotions , 2007, Cyberpsychology Behav. Soc. Netw..

[29]  Elsevier Sdol.  International Journal of Human-Computer Studies , 2009 .

[30]  Cynthia Breazeal,et al.  Affective Learning — A Manifesto , 2004 .

[31]  N. Kalin,et al.  Emotion, plasticity, context, and regulation: perspectives from affective neuroscience. , 2000, Psychological bulletin.

[32]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[33]  A. Dermer,et al.  Relaxing at the perfect beach : influence of auditory stimulation on positive and negative affect in a virtual environment , 2016 .

[34]  A. Rizzo,et al.  Affective outcomes of virtual reality exposure therapy for anxiety and specific phobias: a meta-analysis. , 2008, Journal of behavior therapy and experimental psychiatry.

[35]  Robert P. Sheridan,et al.  Random Forest: A Classification and Regression Tool for Compound Classification and QSAR Modeling , 2003, J. Chem. Inf. Comput. Sci..

[36]  Lakhmi C. Jain,et al.  Advanced Computational Intelligence Paradigms in Healthcare 6. Virtual Reality in Psychotherapy, Rehabilitation, and Assessment , 2011 .

[37]  J. Russell.  A circumplex model of affect. , 1980 .

[38]  Lakhmi C. Jain,et al.  Virtual Reality in Psychotherapy, Rehabilitation, and Neurological Assessment , 2011 .

[39]  Stephen H. Fairclough,et al.  Capturing user engagement via psychophysiology: measures and mechanisms for biocybernetic adaptation , 2013, Int. J. Auton. Adapt. Commun. Syst..

[40]  L. MenezesM.,et al.  Towards emotion recognition for virtual environments , 2017 .

[41]  B. Kedem,et al.  Spectral analysis and discrimination by zero-crossings , 1986, Proceedings of the IEEE.

[42]  Yaacob Sazali,et al.  Classification of human emotion from EEG using discrete wavelet transform , 2010 .

[43]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[44]  Leontios J. Hadjileontiadis,et al.  Emotion Recognition From EEG Using Higher Order Crossings , 2010, IEEE Transactions on Information Technology in Biomedicine.

[45]  Elinda Ai Lim Lee,et al.  An investigation into the effectiveness of virtual reality-based learning , 2011 .

[46]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[47]  Ron Chi-Wai Kwok,et al.  The SAMAL Model for Affective Learning: A multidimensional model incorporating the body, mind and emotion in learning , 2011, DMS.