Emotion recognition of serious game players using a simple brain computer interface

Understanding player's cognitive state and emotional response is necessary to make serious games more challenging and enhance the quality of human-machine interaction and gaming experience. Previous emotion recognition using brain computer interfaces (BCIs) is either relying a large number of wet electrodes or combining both brain signals with other peripheral physiological signals. In this paper, we investigate whether a simple BCI with only a few electrodes can identify basic or complex emotions in more natural settings like playing a game. We performed an experiment with 42 participants, who played a brain-controlled video game wearing a single electrode BCI headset and provided a self-assessed valence/arousal feedback at the end of each trial. By analyzing the data obtained from the self-evaluated questionnaires and the attention and meditation recordings from the BCI device, we introduce an automatic emotion recognition method that classifies four emotional states with the average recognition accuracy 66.04%.

[1]  Ian Pitt,et al.  Evaluating a Brain-Computer Interface to Categorise Human Emotional Response , 2010, 2010 10th IEEE International Conference on Advanced Learning Technologies.

[2]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[3]  Mina Mikhail,et al.  Using minimal number of electrodes for emotion detection using brain signals produced from a new elicitation technique , 2013, Int. J. Auton. Adapt. Commun. Syst..

[4]  Fotis Liarokapis,et al.  Assessing NeuroSky's Usability to Detect Attention Levels in an Assessment Exercise , 2009, HCI.

[5]  Johannes Wagner,et al.  From Physiological Signals to Emotions: Implementing and Comparing Selected Methods for Feature Extraction and Classification , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[6]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[7]  Elisabeth André,et al.  Emotion recognition based on physiological changes in music listening , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[9]  Léon J. M. Rothkrantz,et al.  Emotion recognition using brain activity , 2008, CompSysTech.

[10]  Thomas S. Huang,et al.  Emotion Recognition from Facial Expressions using Multilevel HMM , 2000 .

[11]  Oksam Chae,et al.  Robust Facial Expression Recognition Based on Local Directional Pattern , 2010 .

[12]  Hichem Sahli,et al.  Evaluation of Attention Levels in a Tetris Game Using a Brain Computer Interface , 2013, UMAP.