An Exploration of Machine Learning Methods for Robust Boredom Classification Using EEG and GSR Data

In recent years, affective computing has been actively researched to provide a higher level of emotion-awareness. Numerous studies have been conducted to detect the user’s emotions from physiological data. Among a myriad of target emotions, boredom, in particular, has been suggested to cause not only medical issues but also challenges in various facets of daily life. However, to the best of our knowledge, no previous studies have used electroencephalography (EEG) and galvanic skin response (GSR) together for boredom classification, although these data have potential features for emotion classification. To investigate the combined effect of these features on boredom classification, we collected EEG and GSR data from 28 participants using off-the-shelf sensors. During data acquisition, we used a set of stimuli comprising a video clip designed to elicit boredom and two other video clips of entertaining content. The collected samples were labeled based on the participants’ questionnaire-based testimonies on experienced boredom levels. Using the collected data, we initially trained 30 models with 19 machine learning algorithms and selected the top three candidate classifiers. After tuning the hyperparameters, we validated the final models through 1000 iterations of 10-fold cross validation to increase the robustness of the test results. Our results indicated that a Multilayer Perceptron model performed the best with a mean accuracy of 79.98% (AUC: 0.781). It also revealed the correlation between boredom and the combined features of EEG and GSR. These results can be useful for building accurate affective computing systems and understanding the physiological properties of boredom.

[1]  Jennifer J. Vogel-Walcutt,et al.  The Definition, Assessment, and Mitigation of State Boredom Within Educational Settings: A Comprehensive Review , 2012 .

[2]  Dimitrios Tzovaras,et al.  Automatic Recognition of Boredom in Video Games Using Novel Biosignal Moment-Based Features , 2011, IEEE Transactions on Affective Computing.

[3]  L. Mandolesi Physiology of Behavior , 2002 .

[4]  Bao-Liang Lu,et al.  Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks , 2015, IEEE Transactions on Autonomous Mental Development.

[5]  J. Eastwood,et al.  Development and Validation of the Multidimensional State Boredom Scale , 2013, Assessment.

[6]  Minjuan Wang,et al.  Affective e-Learning: Using "Emotional" Data to Improve Learning in Pervasive Learning Environment , 2009, J. Educ. Technol. Soc..

[7]  Nasser Oroujlou,et al.  Motivation, attitude, and language learning , 2011 .

[8]  Dimitrios Tzovaras,et al.  Identifying Psychophysiological Correlates of Boredom and Negative Mood Induced during HCI , 2010, B-Interface.

[9]  Bao-Liang Lu,et al.  Identifying Stable Patterns over Time for Emotion Recognition from EEG , 2016, IEEE Transactions on Affective Computing.

[10]  Teemu H. Laine,et al.  Machine learning approaches for boredom classification using EEG , 2019, Journal of Ambient Intelligence and Humanized Computing.

[11]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[12]  H. Jasper Report of the committee on methods of clinical examination in electroencephalography , 1958 .

[13]  Angela L. Duckworth,et al.  Boring but important: a self-transcendent purpose for learning fosters academic self-regulation. , 2014, Journal of personality and social psychology.

[14]  Lannie Kanevsky A COMPARATIVE STUDY OF CHILDREN'S LEARNING IN THE ZONE OF PROXIMAL DEVELOPMENT , 1994 .

[15]  P. Lang International affective picture system (IAPS) : affective ratings of pictures and instruction manual , 2005 .

[16]  Cristina Conati,et al.  Predicting Affect from Gaze Data during Interaction with an Intelligent Tutoring System , 2014, Intelligent Tutoring Systems.

[17]  M. Fenske,et al.  The Unengaged Mind , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[18]  Bao-Liang Lu,et al.  Emotion classification based on gamma-band EEG , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[19]  George Tomlinson,et al.  Can Routine Laboratory Tests Discriminate between Severe Acute Respiratory Syndrome and Other Causes of Community-Acquired Pneumonia? , 2005, Clinical infectious diseases : an official publication of the Infectious Diseases Society of America.

[20]  Stephen Ashwal,et al.  Child Neurology in the 20th Century , 2003, Pediatric Research.

[21]  Regan L. Mandryk,et al.  A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies , 2007, Int. J. Hum. Comput. Stud..

[22]  Valerie C. Scanlon,et al.  Essentials of Anatomy and Physiology , 1991 .

[23]  Byoung-Jun Park,et al.  Analysis of physiological signals for recognition of boredom, pain, and surprise emotions , 2015, Journal of Physiological Anthropology.

[24]  J. Russell A circumplex model of affect. , 1980 .

[25]  Heather C. Lench,et al.  On the Function of Boredom , 2013, Behavioral sciences.

[26]  Teemu Henrikki Laine,et al.  Detecting boredom from eye gaze and EEG , 2018, Biomed. Signal Process. Control..

[27]  Daniel R. Fesenmaier,et al.  Measuring Emotions in Real Time , 2015 .

[28]  Scotty D. Craig,et al.  Integrating Affect Sensors in an Intelligent Tutoring System , 2004 .

[29]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[30]  Arthur C. Graesser,et al.  Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments , 2010, Int. J. Hum. Comput. Stud..

[31]  Wolzt,et al.  World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. , 2003, The Journal of the American College of Dentists.

[32]  L. Feldman Variations in the Circumplex Structure of Mood , 1995 .

[33]  Ruimin Shen,et al.  Exploratory Research on an Affective e-Learning Model , 2007 .

[34]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[35]  Yuan-Pin Lin,et al.  EEG-Based Emotion Recognition in Music Listening , 2010, IEEE Transactions on Biomedical Engineering.

[36]  Kristina Höök,et al.  eMoto: emotionally engaging interaction , 2004, Personal and Ubiquitous Computing.

[37]  Martin J Shipley,et al.  Bored to death? , 2010, International journal of epidemiology.

[38]  BakerRyan S. J. d.,et al.  Better to be frustrated than bored , 2010 .

[39]  William T. Thompson,et al.  Effects of Shift Work and Sustained Operations: Operator Performance in Remotely Piloted Aircraft (OP-REPAIR) , 2006 .

[40]  Christiane,et al.  World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. , 2004, Journal international de bioethique = International journal of bioethics.