Recognizing Student Emotions using Brainwaves and Mouse Behavior Data

Brainwaves EEG signals and mouse behavior information are shown to be useful in predicting academic emotions, such as confidence, excitement, frustration and interest. Twenty five college students were asked to use the Aplusix math learning software while their brainwaves signals and mouse behavior number of clicks, duration of each click, distance traveled by the mouse were automatically being captured. It is shown that by combining the extracted features from EEG signals with data representing mouse click behavior, the accuracy in predicting academic emotions substantially increases compared to using only features extracted from EEG signals or just mouse behavior alone. Furthermore, experiments were conducted to assess the prediction accuracy of the system at points during the learning session where several of the extracted features significantly deviate in value from their mean. The experiments confirm that the prediction performance increases as the number of feature values that deviate significantly from the mean increases.

[1]  Robert A. Lordo,et al.  Learning from Data: Concepts, Theory, and Methods , 2001, Technometrics.

[2]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[3]  Jonathan Klein,et al.  Frustrating the user on purpose: a step toward building an affective computer , 2002, Interact. Comput..

[4]  Rosalind W. Picard,et al.  Affective learning companions: strategies for empathetic agents with real-time multimodal affective sensing to foster meta-cognitive and meta-affective approaches to learning, motivation, and perseverance , 2006 .

[5]  Jean-François Nicaud,et al.  The Aplusix-Editor: A New Kind of Software for the Learning of Algebra , 2002, Intelligent Tutoring Systems.

[6]  Arthur C. Graesser,et al.  Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features , 2010, User Modeling and User-Adapted Interaction.

[7]  Claude Frasson,et al.  Predicting Stress Level Variation from Learner Characteristics and Brainwaves , 2009, AIED.

[8]  Saeid Sanei,et al.  EEG signal processing , 2000, Clinical Neurophysiology.

[9]  G. Chanel.  Emotion assessment for affective computing based on brain and peripheral signals , 2009 .

[10]  P. Ekman.  Expression and the Nature of Emotion , 1984 .

[11]  Judith J. Azcarraga,et al.  Use of Personality Profile in Predicting Academic Emotion Based on Brainwaves Signals and Mouse Behavior , 2011, 2011 Third International Conference on Knowledge and Systems Engineering.

[12]  Ashish Kapoor,et al.  Automatic prediction of frustration , 2007, Int. J. Hum. Comput. Stud..

[13]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[14]  P. Zimmermann,et al.  Affective Computing—A Rationale for Measuring Mood With Mouse and Keyboard , 2003, International journal of occupational safety and ergonomics : JOSE.

[15]  Gintautas Dzemyda,et al.  Web-based Biometric Computer Mouse Advisory System to Analyze a User's Emotions and Work Productivity , 2011, Engineering applications of artificial intelligence.

[16]  Claude Frasson,et al.  Using machine learning to predict learner emotional state from brainwaves , 2007, Seventh IEEE International Conference on Advanced Learning Technologies (ICALT 2007).

[17]  P. Ekman,et al.  Approach-withdrawal and cerebral asymmetry: emotional expression and brain physiology. I. , 1990, Journal of personality and social psychology.

[18]  Charalampos Bratsas,et al.  On the Classification of Emotional Biosignals Evoked While Viewing Affective Pictures: An Integrated Data-Mining-Based Approach for Healthcare Applications , 2010, IEEE Transactions on Information Technology in Biomedicine.

[19]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[20]  Ronald H. Stevens,et al.  EEG-Related Changes in Cognitive Workload, Engagement and Distraction as Students Acquire Problem Solving Skills , 2007, User Modeling.

[21]  T. Goetz,et al.  Academic Emotions in Students' Self-Regulated Learning and Achievement: A Program of Qualitative and Quantitative Research , 2002 .

[22]  R. E. Wheeler,et al.  Individual differences in anterior brain asymmetry and fundamental dimensions of emotion. , 1992, Journal of personality and social psychology.

[23]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[24]  Merlin Teodosia Suarez,et al.  Predicting Academic Emotion based on Brainwaves Signals and Mouse Click Behavior , 2011 .

[25]  Kasia Muldner,et al.  Emotion Sensors Go To School , 2009, AIED.

[26]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.