Wemotion: A System to Detect Emotion Using Wristbands and Smartphones

Understanding students’ emotion, especially during the classroom time, can help to improve the positive classroom emotional climate towards promoting academic achievement. Unfortunately, most of the exisiting studies that try to understand the emotion of students have just utilized a questionnaire method to discover the link between the classroom emotional climate and academic achievement. Such methods do not reflect exactly the emotion of students in the real-time mode. There are also other studies that leverage hi-tech technologies (e.g. video camera, sensors, smartphones) to capture data generated by people themselves (e.g. physiological data, facial expression, body postures, human-smartphone interaction) to recognize emotion. Nonetheless, these methods build either a general model for all users or an individual model for a selected user leading to having a weak adaptive ability. In this paper, we introduce Wemotion, a smart-sensing system built by smartphones and wristbands that can not only detect students’ emotion in real-time mode and also evolve to continuously improve the accuracy during the life cycle. The system is evaluated by real data collected from volunteers and compared to several existing methods. The results show that the system works well and satisfies the purpose of our research.

[1]  Murtaza Bulut,et al.  Mobile real-time arousal detection , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Tamás D. Gedeon,et al.  Objective measures, sensors and computational techniques for stress recognition and classification: A survey , 2012, Comput. Methods Programs Biomed..

[3]  Catherine Robertson,et al.  Setting the Mood for Critical Thinking in the Classroom , 2015 .

[4]  Akane Sano,et al.  Stress Recognition Using Wearable Sensors and Mobile Phones , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[5]  Robert A. Sottilare,et al.  Passively Classifying Student Mood and Performance within Intelligent Tutors , 2012, J. Educ. Technol. Soc..

[6]  Anastasios A. Economides,et al.  Mood Recognition during Online Self-Assessment Tests , 2009, IEEE Transactions on Learning Technologies.

[7]  Marc A. Brackett,et al.  Classroom emotional climate, student engagement, and academic achievement , 2012 .

[8]  Sarup R. Mathur,et al.  The Impact of Positive Mood on Learning , 1996 .

[9]  Szwoch Wioleta,et al.  Using physiological signals for emotion recognition , 2013, 2013 6th International Conference on Human System Interactions (HSI).

[10]  Jennifer Healey,et al.  Detecting stress during real-world driving tasks using physiological sensors , 2005, IEEE Transactions on Intelligent Transportation Systems.

[11]  C. Maaoui,et al.  Emotion Recognition through Physiological Signals for Human-Machine Communication , 2010 .

[12]  R. Pekrun.  The impact of emotions on learning and achievement : towards a theory of cognitive/motivational mediators , 1992 .

[13]  N. P. Reddy,et al.  Facial expression (mood) recognition from facial images using committee neural networks , 2009, Biomedical engineering online.

[14]  V. Alexandratos,et al.  Mobile Real-Time Stress Detection , 2014 .

[15]  Kent Larson,et al.  Real-Time Recognition of Physical Activities and Their Intensities Using Wireless Accelerometers and a Heart Rate Monitor , 2007, 2007 11th IEEE International Symposium on Wearable Computers.

[16]  Enzo Pasquale Scilingo,et al.  Wearable Monitoring for Mood Recognition in Bipolar Disorder Based on History-Dependent Long-Term Heart Rate Variability Analysis , 2014, IEEE Journal of Biomedical and Health Informatics.

[17]  Tze Wei Liew,et al.  The Effects of Positive and Negative Mood on Cognition and Motivation in Multimedia Learning Environment , 2016, J. Educ. Technol. Soc..

[18]  Mahesh Sooriyabandara,et al.  HealthyOffice: Mood recognition at work using smartphones and wearable sensors , 2016, 2016 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[19]  Ari Warokka,et al.  The Effects of Positive and Negative Mood on University Students' Learning and Academic Performance: Evidence from Indonesia , 2011 .

[20]  Hao Liu,et al.  Wearable Physiological Sensors Reflect Mental Stress State in Office-Like Situations , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[21]  Joyce H. D. M. Westerink,et al.  Mood Recognition Based on Upper Body Posture and Movement Features , 2011, ACII.

[22]  C. Vinola,et al.  A Survey on Human Emotion Recognition Approaches, Databases and Applications , 2015 .

[23]  Matjaz Gams,et al.  Automatic Detection of Perceived Stress in Campus Students Using Smartphones , 2015, 2015 International Conference on Intelligent Environments.

[24]  Arnab Bag,et al.  A review on emotion recognition using speech , 2017, 2017 International Conference on Inventive Communication and Computational Technologies (ICICCT).

[25]  Haim Hilman Abdullah,et al.  University Students’ Emotion State and Academic Performance: New Insights of Managing Complex Cognitive , 2011 .

[26]  Christophe Ley,et al.  Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median , 2013 .

[27]  Yunxin Liu,et al.  MoodScope: building a mood sensor from smartphone usage patterns , 2013, MobiSys '13.

[28]  Nicu Sebe,et al.  Multimodal approaches for emotion recognition: a survey , 2005, IS&T/SPIE Electronic Imaging.

[29]  Daniel McDuff,et al.  Remote measurement of cognitive stress via heart rate variability , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[30]  Joseph A. O'Sullivan,et al.  ECG Biometric Recognition: A Comparative Analysis , 2012, IEEE Transactions on Information Forensics and Security.