AffectivelyVR: Towards VR Personalized Emotion Recognition

We present AffectivelyVR, a personalized real-time emotion recognition system in Virtual Reality (VR) that enables an emotion-adaptive virtual environment. We used off-the-shelf Electroencephalogram (EEG) and Galvanic Skin Response (GSR) physiological sensors to train user-specific machine learning models while exposing users to affective 360° VR videos. Since emotions are largely dependent on interpersonal experiences and expressed in different ways for different people, we personalize the model instead of generalizing it. By doing this, we achieved an emotion recognition rate of 96.5% using the personalized KNN algorithm, and 83.7% using the generalized SVM algorithm.

[1]  Rosa María Baños,et al.  Changing Induced Moods Via Virtual Reality , 2006, PERSUASIVE.

[2]  Mark Billinghurst,et al.  In AI We Trust: Investigating the Relationship between Biosignals, Trust and Cognitive Load in VR , 2019, VRST.

[3]  Arindam Dey,et al.  Effects of Sharing Physiological States of Players in a Collaborative Virtual Reality Gameplay , 2017, CHI.

[4]  RastgooMohammad Naim,et al.  Evolutionary computation algorithms for feature selection of EEG-based emotion recognition using mobile sensors , 2018 .

[5]  Mark Billinghurst,et al.  Measuring Human Trust in a Virtual Assistant using Physiological Sensing in Virtual Reality , 2020, 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[6]  Vinod Chandran,et al.  Evolutionary computation algorithms for feature selection of EEG-based emotion recognition using mobile sensors , 2018, Expert Syst. Appl..

[7]  Heeseok Oh,et al.  A Deep Cybersickness Predictor Based on Brain Signal Analysis for Virtual Reality Contents , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[8]  C. Darwin.  The Expression of the Emotions in Man and Animals , .

[9]  Bao-Liang Lu,et al.  EEG-Based Emotion Recognition in Listening Music by Using Support Vector Machine and Linear Dynamic System , 2012, ICONIP.

[10]  Anthony J Bishara,et al.  Testing the significance of a correlation with nonnormal data: comparison of Pearson, Spearman, transformation, and resampling approaches. , 2012, Psychological methods.

[11]  Alexander Bertrand,et al.  A generic EEG artifact removal algorithm based on the multi-channel Wiener filter , 2018, Journal of neural engineering.

[12]  Arindam Dey,et al.  Empathic Mixed Reality: Sharing What You Feel and Interacting with What You See , 2017, 2017 International Symposium on Ubiquitous Virtual Reality (ISUVR).

[13]  Mohammad Mansour Riahi Kashani,et al.  Combination of PCA with SMOTE Resampling to Boost the Prediction Rate in Lung Cancer Dataset , 2013, ArXiv.

[14]  Rafael A. Calvo,et al.  Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications , 2010, IEEE Transactions on Affective Computing.

[15]  Yun Jang,et al.  Cybersickness Analysis with EEG Using Deep Learning Algorithms , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[16]  Rosa María Baños,et al.  Immersion and Emotion: Their Impact on the Sense of Presence , 2004, Cyberpsychology Behav. Soc. Netw..

[17]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[18]  Ana L. N. Fred,et al.  A web-based platform for biosignal visualization and annotation , 2013, Multimedia Tools and Applications.

[19]  Mladen Russo,et al.  Wearable Emotion Recognition System based on GSR and PPG Signals , 2017, MMHealth@MM.