Detecting Work Stress in Offices by Combining Unobtrusive Sensors

Employees often report the experience of stress at work. In the SWELL project we investigate how new context aware pervasive systems can support knowledge workers to diminish stress. The focus of this paper is on developing automatic classifiers to infer working conditions and stress related mental states from a multimodal set of sensor data (computer logging, facial expressions, posture and physiology). We address two methodological and applied machine learning challenges: 1) Detecting work stress using several (physically) unobtrusive sensors, and 2) Taking into account individual differences. A comparison of several classification approaches showed that, for our SWELL-KW dataset, neutral and stressful working conditions can be distinguished with 90 percent accuracy by means of SVM. Posture yields most valuable information, followed by facial expressions. Furthermore, we found that the subjective variable ‘mental effort’ can be better predicted from sensor data than, e.g., ‘perceived stress’. A comparison of several regression approaches showed that mental effort can be predicted best by a decision tree (correlation of 0.82). Facial expressions yield most valuable information, followed by posture. We find that especially for estimating mental states it makes sense to address individual differences. When we train models on particular subgroups of similar users, (in almost all cases) a specialized model performs equally well or better than a generic model.

[1]  Giancarlo Fortino,et al.  Middlewares for Smart Objects and Smart Environments: Overview and Comparison , 2014, Internet of Things Based on Smart Objects, Technology, Middleware and Applications.

[2]  Dong-Jun Kim,et al.  Application for the wearable heart activity monitoring system: analysis of the autonomic function of HRV. , 2008, Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference.

[3]  Arthur C. Graesser,et al.  Emote aloud during learning with AutoTutor: Applying the Facial Action Coding System to cognitive–affective states during learning , 2008 .

[4]  Pietro Cipresso,et al.  Electro-Physiological Data Fusion for Stress Detection , 2012, Annual Review of Cybertherapy and Telemedicine.

[5]  Wessel Kraaij,et al.  The SWELL Knowledge Work Dataset for Stress and User Modeling Research , 2014, ICMI.

[6]  Mark A. Neerincx,et al.  Modeling the Cognitive Task Load and Performance of Naval Operators , 2009, HCI.

[7]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[8]  Rich Caruana,et al.  An empirical comparison of supervised learning algorithms , 2006, ICML.

[9]  Annemiek Drunen,et al.  Exploring workload and attention measurements with uLog mouse data , 2009, Behavior research methods.

[10]  Günther Palm,et al.  A generic framework for the inference of user states in human computer interaction , 2012, Journal on Multimodal User Interfaces.

[11]  Wessel Kraaij,et al.  Visual Analytics of Work Behavior Data - Insights on Individual Differences , 2015, EuroVis.

[12]  Regan L. Mandryk,et al.  Identifying emotional states using keystroke dynamics , 2011, CHI.

[13]  G. Fortino,et al.  SPINE-HRV: A BSN-Based Toolkit for Heart Rate Variability Analysis in the Time-Domain , 2010 .

[14]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[15]  Ashish Kapoor,et al.  Multimodal affect recognition in learning environments , 2005, ACM Multimedia.

[16]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[17]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[18]  Roozbeh Jafari,et al.  Enabling Effective Programming and Flexible Management of Efficient Body Sensor Network Applications , 2013, IEEE Transactions on Human-Machine Systems.

[19]  F. Mokhayeri,et al.  Mental stress detection using physiological signals based on soft computing techniques , 2011, 2011 18th Iranian Conference of Biomedical Engineering (ICBME).

[20]  Saskia Koldijk Automatic recognition of context and stress to support knowledge workers , 2012, ECCE.

[21]  Dimitris N. Metaxas,et al.  Optical computer recognition of facial expressions associated with stress induced by performance demands. , 2005, Aviation, space, and environmental medicine.

[22]  D. Zapf Stress-oriented Analysis of Computerized Office Work , 1993 .

[23]  Alex Pentland,et al.  Human computing and machine understanding of human behavior: a survey , 2006, ICMI '06.

[24]  Robert M. Hierons,et al.  Towards a Computer Interaction-Based Mood Measurement Instrument , 2008 .

[25]  Andrew Sears,et al.  Automated stress detection using keystroke and linguistic features: An exploratory study , 2009, Int. J. Hum. Comput. Stud..

[26]  Marko Munih,et al.  A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing , 2012, Interact. Comput..

[27]  Gerhard Tröster,et al.  Discriminating Stress From Cognitive Load Using a Wearable EDA Device , 2010, IEEE Transactions on Information Technology in Biomedicine.

[28]  Wessel Kraaij,et al.  Unobtrusive Monitoring of Knowledge Workers for Stress Self-regulation , 2013, UMAP.

[29]  Mykola Pechenizkiy,et al.  Stess@Work: from measuring stress to its understanding, prediction and handling with personalized coaching , 2012, IHI '12.

[30]  Gerhard Tröster,et al.  Monitoring of mental workload levels during an everyday life office-work scenario , 2013, Personal and Ubiquitous Computing.

[31]  Wessel Kraaij,et al.  Activity-logging for self-coaching of knowledge workers , 2011, ECIR 2011.