Long Short-Term Network Based Unobtrusive Perceived Workload Monitoring with Consumer Grade Smartwatches in the Wild

Continuous high perceived workload has a negative impact on the individual's well-being. Prior works focused on detecting the workload with medical-grade wearable systems in the restricted settings, and the effect of applying deep learning techniques for perceived workload detection in the wild settings is not investigated. We present an unobtrusive, comfortable, pervasive and affordable Long Short-Term Memory Network based continuous workload monitoring system based on a smartwatch application that monitors the perceived workload of individuals in the wild. We make use of modern consumer-grade smartwatches. We have recorded physiological data from daily life with perceived workload questionnaires from subjects in their real-life environments over a month. The model was trained and evaluated with the daily-life physiological data coming from different days which makes it robust to daily changes in the heart rate variability, that we use with accelerometer features to asses low and high workload. Our system has the capability of removing motion-related artifacts and detecting perceived workload by using traditional and deep classifiers. We discussed the problems related to in the wild applications with the consumer-grade smartwatches. We showed that Long Short-Term Memory Network outperforms traditional classifiers on discrimination of low and high workload with smartwatches in the wild.

[1]  Serkan Bugur,et al.  Sign sentence recognition with smart watches , 2017, 2017 25th Signal Processing and Communications Applications Conference (SIU).

[2]  Akane Sano,et al.  Identifying Objective Physiological Markers and Modifiable Behaviors for Self-Reported Stress and Mental Health Status Using Wearable Sensors and Mobile Phones: Observational Study , 2018, Journal of medical Internet research.

[3]  Cem Ersoy,et al.  Continuous Stress Detection Using Wearable Sensors in Real Life: Algorithmic Programming Contest Case Study , 2019, Sensors.

[4]  Rosalind W. Picard.  Automating the Recognition of Stress and Emotion: From Lab to Real-World Impact , 2016, IEEE Multim..

[5]  Nilanjan Sarkar,et al.  EEG-Based Affect and Workload Recognition in a Virtual Driving Environment for ASD Intervention , 2018, IEEE Transactions on Biomedical Engineering.

[6]  Cem Ersoy,et al.  Smart watch based stress detection in real life , 2018 .

[7]  Erhardt Barth,et al.  Recurrent Dropout without Memory Loss , 2016, COLING.

[8]  Nuran Bayram,et al.  The prevalence and socio-demographic correlations of depression, anxiety and stress among a group of university students , 2008, Social Psychiatry and Psychiatric Epidemiology.

[9]  P. Ekman.  Differential communication of affect by head and body cues. , 1965, Journal of personality and social psychology.

[10]  R Darin Ellis,et al.  NASA TLX: Software for assessing subjective mental workload , 2009, Behavior research methods.

[11]  Delbert M. Nebeker,et al.  Workload and social support: Effects on performance and stress , 1999 .

[12]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[13]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[14]  Xiao-Hua Yu,et al.  An adaptive filtering approach for electrocardiogram (ECG) signal noise reduction using neural networks , 2013, Neurocomputing.

[15]  Archan Misra,et al.  Breathing-Based Authentication on Resource-Constrained IoT Devices using Recurrent Neural Networks , 2018, Computer.

[16]  Evangelos Karapanos,et al.  Workload management through glanceable feedback: The role of heart rate variability , 2016, 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom).

[17]  Ian H. Witten,et al.  WEKA: a machine learning workbench , 1994, Proceedings of ANZIIS '94 - Australian New Zealnd Intelligent Information Systems Conference.

[18]  Reda A. El-Khoribi,et al.  Emotion Recognition based on EEG using LSTM Recurrent Neural Network , 2017 .

[19]  Naif Alajlan,et al.  Deep learning approach for active classification of electrocardiogram signals , 2016, Inf. Sci..

[20]  Matjaz Gams,et al.  Continuous stress detection using a wrist device: in laboratory and real life , 2016, UbiComp Adjunct.

[21]  E. Hari Krishna,et al.  A Novel Approach for Motion Artifact Reduction in PPG Signals Based on AS-LMS Adaptive Filter , 2012, IEEE Transactions on Instrumentation and Measurement.

[22]  Cem Ersoy,et al.  Stress detection in daily life scenarios using smart phones and wearable sensors: A survey , 2019, J. Biomed. Informatics.

[23]  T. Tang,et al.  Activity awareness can improve continuous stress detection in galvanic skin response , 2014, IEEE SENSORS 2014 Proceedings.

[24]  Jan Ole Johanssen,et al.  Employing Consumer Wearables to Detect Office Workers' Cognitive Load for Interruption Management , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[25]  Adrian Basarab,et al.  Towards an automatic early stress recognition system for office environments based on multimodal measurements: A review , 2016, J. Biomed. Informatics.

[26]  Andrew R. A. Conway,et al.  The invention of n-back: An extremely brief history , 2016 .

[27]  Ethem Alpaydin,et al.  Introduction to machine learning , 2004, Adaptive computation and machine learning.

[28]  Matjaz Gams,et al.  Monitoring stress with a wrist device using context , 2017, J. Biomed. Informatics.

[29]  Gerhard Tröster,et al.  Monitoring of mental workload levels during an everyday life office-work scenario , 2013, Personal and Ubiquitous Computing.

[30]  David Sharek,et al.  A Useable, Online NASA-TLX Tool , 2011 .