Machine learning approaches for boredom classification using EEG

Recently, commercial physiological sensors and computing devices have become cheaper and more accessible, while computer systems have become increasingly aware of their contexts, including but not limited to users’ emotions. Consequently, many studies on emotion recognition have been conducted. However, boredom has received relatively little attention as a target emotion due to its diverse nature. Moreover, only a few researchers have tried classifying boredom using electroencephalogram (EEG). In this study, to perform this classification, we first reviewed studies that tried classifying emotions using EEG. Further, we designed and executed an experiment, which used a video stimulus to evoke boredom and non-boredom, and collected EEG data from 28 Korean adult participants. After collecting the data, we extracted its absolute band power, normalized absolute band power, differential entropy, differential asymmetry, and rational asymmetry using EEG, and trained these on three machine learning algorithms: support vector machine, random forest, and k-nearest neighbors (k-NN). We validated the performance of each training model with 10-fold cross validation. As a result, we achieved the highest accuracy of 86.73% using k-NN. The findings of this study can be of interest to researchers working on emotion recognition, physiological signal processing, machine learning, and emotion-aware system development.

[1]  P. Lang International affective picture system (IAPS) : affective ratings of pictures and instruction manual , 2005 .

[2]  Loïc Kessous,et al.  Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech , 2008, Affect and Emotion in Human-Computer Interaction.

[3]  Lannie Kanevsky A COMPARATIVE STUDY OF CHILDREN'S LEARNING IN THE ZONE OF PROXIMAL DEVELOPMENT , 1994 .

[4]  Saeid Sanei,et al.  EEG signal processing , 2000, Clinical Neurophysiology.

[5]  Joy Kreeft Peyton,et al.  Collaborative writing in multiple discourse contexts , 1994, Computer Supported Cooperative Work (CSCW).

[6]  Nasser Oroujlou,et al.  Motivation, attitude, and language learning , 2011 .

[7]  J. Bhattacharya,et al.  Crossmodal transfer of emotion by music , 2009, Neuroscience Letters.

[8]  N. Fox,et al.  Asymmetrical brain activity discriminates between positive and negative affective stimuli in human infants. , 1982, Science.

[9]  Arthur C. Graesser,et al.  Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments , 2010, Int. J. Hum. Comput. Stud..

[10]  Léon J. M. Rothkrantz,et al.  Emotion recognition using brain activity , 2008, CompSysTech.

[11]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[12]  M. Fenske,et al.  The Unengaged Mind , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[13]  W. Wundt,et al.  Outline of psychology. , 1897 .

[14]  Frank Rosenblatt,et al.  PRINCIPLES OF NEURODYNAMICS. PERCEPTRONS AND THE THEORY OF BRAIN MECHANISMS , 1963 .

[15]  Leontios J. Hadjileontiadis,et al.  Emotion Recognition from Brain Signals Using Hybrid Adaptive Filtering and Higher Order Crossings Analysis , 2010, IEEE Transactions on Affective Computing.

[16]  Regan L. Mandryk,et al.  A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies , 2007, Int. J. Hum. Comput. Stud..

[17]  R. Davidson Anterior cerebral asymmetry and the nature of emotion , 1992, Brain and Cognition.

[18]  Bao-Liang Lu,et al.  Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks , 2015, IEEE Transactions on Autonomous Mental Development.

[19]  Akira Yasukouchi Reviewer Acknowledgment 2014 , 2015, Journal of Physiological Anthropology.

[20]  Zheng Li,et al.  Implementing emotion-based user-aware e-learning , 2009, CHI Extended Abstracts.

[21]  Dimitrios Tzovaras,et al.  Automatic Recognition of Boredom in Video Games Using Novel Biosignal Moment-Based Features , 2011, IEEE Transactions on Affective Computing.

[22]  Jennifer J. Vogel-Walcutt,et al.  The Definition, Assessment, and Mitigation of State Boredom Within Educational Settings: A Comprehensive Review , 2012 .

[23]  Bao-Liang Lu,et al.  Identifying Stable Patterns over Time for Emotion Recognition from EEG , 2016, IEEE Transactions on Affective Computing.

[24]  J. Russell A circumplex model of affect. , 1980 .

[25]  Tai-Shih Chi,et al.  Robust emotion recognition by spectro-temporal modulation statistic features , 2011, Journal of Ambient Intelligence and Humanized Computing.

[26]  M. Cole Cultural psychology: a once and future discipline? , 1996, Nebraska Symposium on Motivation. Nebraska Symposium on Motivation.

[27]  A. Damasio,et al.  Emotion, decision making and the orbitofrontal cortex. , 2000, Cerebral cortex.

[28]  Chien-Cheng Lee,et al.  An improved boosting algorithm and its application to facial emotion recognition , 2011, Journal of Ambient Intelligence and Humanized Computing.

[29]  Tjeerd Plomp,et al.  The integration of computer use in education , 1992 .

[30]  Bertram C. Bruce,et al.  Electronic Quills: A Situated Evaluation of Using Computers for Writing in Classrooms. Technology in Education Series. , 1992 .

[31]  Byoung-Jun Park,et al.  Analysis of physiological signals for recognition of boredom, pain, and surprise emotions , 2015, Journal of Physiological Anthropology.

[32]  Seyyed Mohammad Reza Hashemi,et al.  Classification of EEG-based emotion for BCI applications , 2017, 2017 Artificial Intelligence and Robotics (IRANOPEN).

[33]  Kwee-Bo Sim,et al.  Brain-wave Analysis using fMRI, TRS and EEG for Human Emotion Recognition , 2007 .

[34]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[35]  Bao-Liang Lu,et al.  EEG-based emotion recognition during watching movies , 2011, 2011 5th International IEEE/EMBS Conference on Neural Engineering.

[36]  Dongkyoo Shin,et al.  The Classification Algorithm of Users` Emotion Using Brain-Wave , 2014 .

[37]  H. Jasper Report of the committee on methods of clinical examination in electroencephalography , 1958 .

[38]  Roman Bednarik,et al.  What do you want to do next: a novel approach for intent prediction in gaze-based interaction , 2012, ETRA.

[39]  Carl Bereiter,et al.  Education and Mind in the Knowledge Age , 2002 .

[40]  Sabrina Boll,et al.  Oxytocin improves facial emotion recognition in young adults with antisocial personality disorder , 2017, Psychoneuroendocrinology.

[41]  Antonio Camurri,et al.  Technique for automatic emotion recognition by body gesture analysis , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[42]  Cecilia Ovesdotter Alm,et al.  Emotions from Text: Machine Learning for Text-based Emotion Prediction , 2005, HLT.

[43]  N. Altman An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression , 1992 .

[44]  L. Feldman Variations in the Circumplex Structure of Mood , 1995 .

[45]  Hatice Gunes,et al.  Bi-modal emotion recognition from expressive face and body gestures , 2007, J. Netw. Comput. Appl..

[46]  T. Baumgartner,et al.  From emotion perception to emotion experience: emotions evoked by pictures and classical music. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[47]  M.H. Moradi,et al.  Emotion detection using brain and peripheral signals , 2008, 2008 Cairo International Biomedical Engineering Conference.

[48]  Rosalind W. Picard Affective Computing , 1997 .

[49]  Say Wei Foo,et al.  Speech emotion recognition using hidden Markov models , 2003, Speech Commun..

[50]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Yuan-Pin Lin,et al.  EEG-Based Emotion Recognition in Music Listening , 2010, IEEE Transactions on Biomedical Engineering.

[52]  Kristina Höök,et al.  eMoto: emotionally engaging interaction , 2004, Personal and Ubiquitous Computing.

[53]  Noriko Nagata,et al.  EEG Correlates of the Flow State: A Combination of Increased Frontal Theta and Moderate Frontocentral Alpha Rhythm in the Mental Arithmetic Task , 2018, Front. Psychol..

[54]  Min-Chai Hsieh,et al.  An emotion recognition mechanism based on the combination of mutual information and semantic clues , 2012, J. Ambient Intell. Humaniz. Comput..

[55]  S. C. Neoh,et al.  A Micro-GA Embedded PSO Feature Selection Approach to Intelligent Facial Emotion Recognition , 2017, IEEE Transactions on Cybernetics.

[56]  Yuan-Pin Lin,et al.  Support vector machine for EEG signal classification during listening to emotional music , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[57]  S. Yaacob,et al.  Lifting scheme for human emotion recognition using EEG , 2008, 2008 International Symposium on Information Technology.

[58]  Yuan-Pin Lin,et al.  EEG-based emotion recognition in music listening: A comparison of schemes for multiclass support vector machine , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[59]  Robert M. Haralick,et al.  Feature normalization and likelihood-based similarity measures for image retrieval , 2001, Pattern Recognit. Lett..

[60]  Cristina Conati,et al.  Predicting Affect from Gaze Data during Interaction with an Intelligent Tutoring System , 2014, Intelligent Tutoring Systems.

[61]  Ahmed Bouridane,et al.  Emotion recognition from scrambled facial images via many graph embedding , 2017, Pattern Recognit..

[62]  Angela L. Duckworth,et al.  Boring but important: a self-transcendent purpose for learning fosters academic self-regulation. , 2014, Journal of personality and social psychology.

[63]  Bao-Liang Lu,et al.  Emotion classification based on gamma-band EEG , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[64]  Chung-Hsien Wu,et al.  Emotion recognition from text using semantic labels and separable mixture models , 2006, TALIP.

[65]  L. Trainor,et al.  Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions , 2001 .

[66]  Ze-Jing Chuang,et al.  Multi-Modal Emotion Recognition from Speech and Text , 2004, ROCLING/IJCLCLP.

[67]  J. Eastwood,et al.  Development and Validation of the Multidimensional State Boredom Scale , 2013, Assessment.

[68]  Minjuan Wang,et al.  Affective e-Learning: Using "Emotional" Data to Improve Learning in Pervasive Learning Environment , 2009, J. Educ. Technol. Soc..

[69]  Robert Oostenveld,et al.  The five percent electrode system for high-resolution EEG and ERP measurements , 2001, Clinical Neurophysiology.

[70]  Xin Wang,et al.  Human emotion and cognition recognition from body language of the head using soft computing techniques , 2012, Journal of Ambient Intelligence and Humanized Computing.

[71]  Zhigang Deng,et al.  Analysis of emotion recognition using facial expressions, speech and multimodal information , 2004, ICMI '04.

[72]  Holger Hoffmann,et al.  Repeatability of facial electromyography (EMG) activity over corrugator supercilii and zygomaticus major on differentiating various emotions , 2011, Journal of Ambient Intelligence and Humanized Computing.

[73]  Teemu Henrikki Laine,et al.  Detecting boredom from eye gaze and EEG , 2018, Biomed. Signal Process. Control..

[74]  Marlene Scardamalia,et al.  Computer Support for Knowledge-Building Communities , 1994 .

[75]  Aravind E. Vijayan,et al.  EEG-Based Emotion Recognition Using Statistical Measures and Auto-Regressive Modeling , 2015, 2015 IEEE International Conference on Computational Intelligence & Communication Technology.

[76]  Leontios J. Hadjileontiadis,et al.  A Novel Emotion Elicitation Index Using Frontal Brain Asymmetry for Enhanced EEG-Based Emotion Recognition , 2011, IEEE Transactions on Information Technology in Biomedicine.

[77]  Christine L. Lisetti Affective computing , 1998, Pattern Analysis and Applications.

[78]  D. N. Tibarewala,et al.  Negative emotion recognition from stimulated EEG signals , 2012, 2012 Third International Conference on Computing, Communication and Networking Technologies (ICCCNT'12).

[79]  Song Ki Sang,et al.  The effects of affective feedbacks according to the learner's emotions in e-learning , 2007 .

[80]  Guillaume Chanel,et al.  Emotion Assessment: Arousal Evaluation Using EEG's and Peripheral Physiological Signals , 2006, MRCS.