Annotation and prediction of stress and workload from physiological and inertial signals

Continuous daily stress and high workload can have negative effects on individuals' physical and mental well-being. It has been shown that physiological signals may support the prediction of stress and workload. However, previous research is limited by the low diversity of signals concurring to such predictive tasks and controlled experimental design. In this paper we present 1) a pipeline for continuous and real-life acquisition of physiological and inertial signals 2) a mobile agent application for on-the-go event annotation and 3) an end-to-end signal processing and classification system for stress and workload from diverse signal streams. We study physiological signals such as Galvanic Skin Response (GSR), Skin Temperature (ST), Inter Beat Interval (IBI) and Blood Volume Pulse (BVP) collected using a non-invasive wearable device; and inertial signals collected from accelerometer and gyroscope sensors. We combine them with subjects' inputs (e.g. event tagging) acquired using the agent application, and their emotion regulation scores. In our experiments we explore signal combination and selection techniques for stress and workload prediction from subjects whose signals have been recorded continuously during their daily life. The end-to-end classification system is described for feature extraction, signal artifact removal, and classification. We show that a combination of physiological, inertial and user event signals provides accurate prediction of stress for real-life users and signals.

[1]  Otto B. Walter,et al.  The Perceived Stress Questionnaire (PSQ) Reconsidered: Validation and Reference Values From Different Clinical and Healthy Adult Samples , 2005, Psychosomatic medicine.

[2]  B. Arnrich,et al.  Towards Continuous Monitoring of Mental Workload , 2010 .

[3]  Peter Nickel,et al.  Sensitivity and Diagnosticity of the 0.1-Hz Component of Heart Rate Variability as an Indicator of Mental Workload , 2003, Hum. Factors.

[4]  H. Kasai,et al.  Relationships between perceived workload, stress and oxidative DNA damage , 2001, International archives of occupational and environmental health.

[5]  Marek Penhaker,et al.  Mobile Phone Application in Biotelemetry , .

[6]  J. Gross The Emerging Field of Emotion Regulation: An Integrative Review , 1998 .

[7]  John A. Detre,et al.  Imaging brain fatigue from sustained mental workload: An ASL perfusion study of the time-on-task effect , 2010, NeuroImage.

[8]  Arindam Ghosh,et al.  Recognizing Human Activities from Smartphone Sensor Signals , 2014, ACM Multimedia.

[9]  A. Loewy,et al.  Central Command Neurons of the Sympathetic Nervous System: Basis of the Fight-or-Flight Response , 1995, Science.

[10]  Javier Hernandez,et al.  Call Center Stress Recognition with Person-Specific Models , 2011, ACII.

[11]  N. Meshkati Heart Rate Variability and Mental Workload Assessment , 1988 .

[12]  James J. Gross,et al.  A Spanish Adaptation of the Emotion Regulation Questionnaire , 2010 .

[13]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[14]  Adisorn Sirikham Abnormal heart rate detection device warning via mobile phone network , 2011, 2011 International Conference on Multimedia Computing and Systems.

[15]  Jennifer Healey,et al.  Detecting stress during real-world driving tasks using physiological sensors , 2005, IEEE Transactions on Intelligent Transportation Systems.