Detecting Moments of Stress from Measurements of Wearable Physiological Sensors

There is a rich repertoire of methods for stress detection using various physiological signals and algorithms. However, there is still a gap in research efforts moving from laboratory studies to real-world settings. A small number of research has verified when a physiological response is a reaction to an extrinsic stimulus of the participant’s environment in real-world settings. Typically, physiological signals are correlated with the spatial characteristics of the physical environment, supported by video records or interviews. The present research aims to bridge the gap between laboratory settings and real-world field studies by introducing a new algorithm that leverages the capabilities of wearable physiological sensors to detect moments of stress (MOS). We propose a rule-based algorithm based on galvanic skin response and skin temperature, combing empirical findings with expert knowledge to ensure transferability between laboratory settings and real-world field studies. To verify our algorithm, we carried out a laboratory experiment to create a “gold standard” of physiological responses to stressors. We validated the algorithm in real-world field studies using a mixed-method approach by spatially correlating the participant’s perceived stress, geo-located questionnaires, and the corresponding real-world situation from the video. Results show that the algorithm detects MOS with 84% accuracy, showing high correlations between measured (by wearable sensors), reported (by questionnaires and eDiary entries), and recorded (by video) stress events. The urban stressors that were identified in the real-world studies originate from traffic congestion, dangerous driving situations, and crowded areas such as tourist attractions. The presented research can enhance stress detection in real life and may thus foster a better understanding of circumstances that bring about physiological stress in humans.

[1]  M. Figliozzi,et al.  Modeling the impact of traffic conditions and bicycle facilities on cyclists’ on-road stress levels , 2018, Transportation Research Part F: Traffic Psychology and Behaviour.

[2]  Martin Dijst,et al.  Wearables and Location Tracking Technologies for Mental-State Sensing in Outdoor Environments , 2019, The Professional Geographer.

[3]  Isabelle Bichindaritz,et al.  Machine learning for stress detection from ECG signals in automobile drivers , 2015, 2015 IEEE International Conference on Big Data (Big Data).

[4]  Aditya Jain,et al.  A comparative study of visual and auditory reaction times on the basis of gender and physical activity levels of medical first year students , 2015, International journal of applied & basic medical research.

[5]  Wei Gao,et al.  Pervasive and unobtrusive emotion sensing for human mental health , 2013, 2013 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops.

[6]  S. C. Mukhopadhyay,et al.  Towards the smart sensors based human emotion recognition , 2012, 2012 IEEE International Instrumentation and Measurement Technology Conference Proceedings.

[7]  S. A. Hosseini,et al.  Emotional Stress Recognition System Using EEG and Psychophysiological Signals: Using New Labelling Process of EEG Signals in Emotional Stress State , 2010, 2010 International Conference on Biomedical Engineering and Computer Science.

[8]  M. Dawson,et al.  The electrodermal system , 2007 .

[9]  Francesco Aletta,et al.  Handbook of Research on Perception-Driven Approaches to Urban Assessment and Design , 2018 .

[10]  J. Ord,et al.  Local Spatial Autocorrelation Statistics: Distributional Issues and an Application , 2010 .

[11]  Bernd Resch,et al.  Spatial Analysis of Moments of Stress Derived from Wearable Sensor Data , 2019, Advances in Cartography and GIScience of the ICA.

[12]  A. Getis The Analysis of Spatial Association by Use of Distance Statistics , 2010 .

[13]  D. Fotiadis,et al.  An integrated telemedicine platform for the assessment of affective physiological states , 2006, Diagnostic pathology.

[14]  Sylvia D. Kreibig,et al.  An affective computing approach to physiological emotion specificity: toward subject-independent and stimulus-independent classification of film-induced emotions. , 2011, Psychophysiology.

[15]  Mobyen Uddin Ahmed,et al.  Intelligent Signal Analysis Using Case-Based Reasoning for Decision Support in Stress Management , 2010 .

[16]  Gonzalo Bailador,et al.  Stress detection by means of stress physiological template , 2011, 2011 Third World Congress on Nature and Biologically Inspired Computing.

[17]  J. Wyatt,et al.  Basic concepts in medical informatics , 2002, Journal of epidemiology and community health.

[18]  F. Shaffer,et al.  An Overview of Heart Rate Variability Metrics and Norms , 2017, Front. Public Health.

[19]  G. Breithardt,et al.  Heart rate variability: standards of measurement, physiological interpretation and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. , 1996 .

[20]  Simon Ollander Wearable Sensor Data Fusion for Human Stress Estimation , 2015 .

[21]  Filippo Cavallo,et al.  A Wearable System for Stress Detection Through Physiological Data Analysis , 2016, ForItAAL.

[22]  Kavallur Gopi Smitha,et al.  EEG based stress level identification , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[23]  F. Wilhelm,et al.  Emotions beyond the laboratory: Theoretical fundaments, study design, and analytic strategies for advanced ambulatory assessment , 2010, Biological Psychology.

[24]  A. Malliani,et al.  Heart rate variability. Standards of measurement, physiological interpretation, and clinical use , 1996 .

[25]  Sylvia D. Kreibig,et al.  Cardiovascular, electrodermal, and respiratory response patterns to fear- and sadness-inducing films. , 2007, Psychophysiology.

[26]  O. V. Ramana Murthy,et al.  Stress Detection in Working People , 2017 .

[27]  Kristof Van Laerhoven,et al.  Wearable affect and stress recognition: A review , 2018, ArXiv.

[28]  Gerhard Tröster,et al.  Discriminating Stress From Cognitive Load Using a Wearable EDA Device , 2010, IEEE Transactions on Information Technology in Biomedicine.

[29]  Zhiwei Zhu,et al.  A Real-Time Human Stress Monitoring System Using Dynamic Bayesian Network , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[30]  Emre Ertin,et al.  cStress: towards a gold standard for continuous stress assessment in the mobile environment , 2015, UbiComp.

[31]  George P. Chrousos,et al.  Mechanisms of Physical and Emotional Stress , 1988, Advances in Experimental Medicine and Biology.

[32]  Mi-hee Lee,et al.  Development stress monitoring system based on personal digital assistant (PDA) , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[33]  Katharine S. Willis,et al.  Engaging the Senses: The Potential of Emotional Data as a New Information Layer in Urban Planning , 2018 .

[34]  Emre Ertin,et al.  Continuous inference of psychological stress from sensory measurements collected in the natural environment , 2011, Proceedings of the 10th ACM/IEEE International Conference on Information Processing in Sensor Networks.

[35]  Gerhard Tröster,et al.  Effect of Movements on the Electrodermal Response after a Startle Event , 2008, Methods of Information in Medicine.

[36]  Katharine S. Willis,et al.  Engaging the Senses: The Potential of Emotional Data for Participation in Urban Planning , 2018, Urban Science.

[37]  Peter Zeile,et al.  Human Sensory Assessment Methods in Urban Planning - a Case Study in Alexandria , 2013 .

[38]  O Barnea,et al.  Spontaneous skin temperature oscillations in normal human subjects. , 1997, The American journal of physiology.

[39]  Marilyn A. Uy,et al.  The Body and the Brain: Measuring Skin Conductance Responses to Understand the Emotional Experience , 2019 .

[40]  Sylvia D. Kreibig,et al.  Attend or defend? Sex differences in behavioral, autonomic, and respiratory response patterns to emotion–eliciting films , 2017, Biological Psychology.

[41]  M. Wright,et al.  Effects of auditory stimuli on electrical activity in the brain during cycle ergometry , 2017, Physiology & Behavior.

[42]  Angel Jiménez Molina,et al.  Using Psychophysiological Sensors to Assess Mental Workload During Web Browsing , 2018, Sensors.

[43]  George Fink,et al.  Stress: Definition and History , 2009 .

[44]  Adrian Basarab,et al.  Towards an automatic early stress recognition system for office environments based on multimodal measurements: A review , 2016, J. Biomed. Informatics.

[45]  José Manuel Pastor,et al.  Electrodermal Activity Sensor for Classification of Calm/Distress Condition , 2017, Sensors.

[46]  Kikuo Asai,et al.  The Role of Head-Up Display in Computer- Assisted Instruction , 2008, ArXiv.

[47]  Robbie T. Nakatsu,et al.  Rule‐Based Expert Systems , 2009 .

[48]  Fernando Seoane,et al.  Wearable Biomedical Measurement Systems for Assessment of Mental Stress of Combatants in Real Time , 2014, Sensors.

[49]  P. Gupta,et al.  Detection of Stress Using Biosensors , 2018 .

[50]  Matjaz Gams,et al.  Continuous stress detection using a wrist device: in laboratory and real life , 2016, UbiComp Adjunct.

[51]  R. F. Thompson,et al.  Habituation: a model phenomenon for the study of neuronal substrates of behavior. , 1966, Psychological review.

[52]  Jin Zhang,et al.  deStress: Mobile and remote stress monitoring, alleviation, and management platform , 2012, 2012 IEEE Global Communications Conference (GLOBECOM).

[53]  Stefan Schmidt,et al.  Electrodermal Activity (Eda) -- State-of-the-Art Measurement and Techniques for Parapsychological Purposes , 1999 .

[54]  Armando Barreto,et al.  Stress Recognition Using Non-invasive Technology , 2006, FLAIRS.

[55]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[56]  H. Staats,et al.  Preference for Nature in Urbanized Societies: Stress, Restoration, and the Pursuit of Sustainability , 2007 .

[57]  P. Rishi,et al.  Urban Environmental Stress and Behavioral Adaptation in Bhopal City of India , 2012 .

[58]  Hao Liu,et al.  Wearable Physiological Sensors Reflect Mental Stress State in Office-Like Situations , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[59]  Gintautas Dzemyda,et al.  Web-based Biometric Computer Mouse Advisory System to Analyze a User's Emotions and Work Productivity , 2011, Eng. Appl. Artif. Intell..

[60]  Mohammad Soleymani,et al.  Highlight Detection in Movie Scenes Through Inter-users, Physiological Linkage , 2013, Social Media Retrieval.

[61]  Jennifer Healey,et al.  Detecting stress during real-world driving tasks using physiological sensors , 2005, IEEE Transactions on Intelligent Transportation Systems.

[62]  W. Tobler A Computer Movie Simulating Urban Growth in the Detroit Region , 1970 .

[63]  W. Boucsein Electrodermal activity, 2nd ed. , 2012 .

[64]  M. Baldo,et al.  Effects of intensity and positional predictability of a visual stimulus on simple reaction time , 2011, Neuroscience Letters.

[65]  Bernd Resch,et al.  A Geoprivacy by Design Guideline for Research Campaigns That Use Participatory Sensing Data , 2018, Journal of empirical research on human research ethics : JERHRE.

[66]  M. H. Schut,et al.  Computing emotion awareness through galvanic skin response and facial electromyography , 2008 .

[67]  Bo Zhang,et al.  Stress Recognition from Heterogeneous Data , 2016 .

[68]  W. F. Prokasy,et al.  Electrodermal Activity in Psychological Research , 1973 .

[69]  Thomas Blaschke,et al.  Contextual Sensing: Integrating Contextual Information with Human and Technical Geo-Sensor Information for Smart Cities , 2015, Sensors.

[70]  A. Bond,et al.  CHARACTERISTICS OF GALVANIC SKIN RESPONSE IN ANXIETY STATES , 1975 .

[71]  Joris C Verster,et al.  The effect of stress on core and peripheral body temperature in humans , 2013, Stress.

[72]  M. Benedek,et al.  Decomposition of skin conductance data by means of nonnegative deconvolution , 2010, Psychophysiology.

[73]  Terence K. L. Hui,et al.  Coverage of Emotion Recognition for Common Wearable Biosensors , 2018, Biosensors.

[74]  László Lengyel Validating Rule-based Algorithms , 2015 .

[75]  Bernd Resch,et al.  Defining and assessing walkability: a concept for an integrated approach using surveys, biosensors and geospatial analysis , 2019, Urban Development Issues.

[76]  Matjaz Gams,et al.  Automatic Detection of Perceived Stress in Campus Students Using Smartphones , 2015, 2015 International Conference on Intelligent Environments.

[77]  Tamás D. Gedeon,et al.  Hybrid Genetic Algorithms for Stress Recognition in Reading , 2013, EvoBIO.

[78]  Developing an affective working companion utilising GSR data , 2011 .

[79]  J. Uttley,et al.  Eye-Tracking in the Real World: Insights About the Urban Environment , 2018 .

[80]  Boreom Lee,et al.  Detection of Stress Levels from Biosignals Measured in Virtual Reality Environments Using a Kernel-Based Extreme Learning Machine , 2017, Sensors.

[81]  Begoña García Zapirain,et al.  A Stress Sensor Based on Galvanic Skin Response (GSR) Controlled by ZigBee , 2012, Sensors.

[82]  Sazali Yaacob,et al.  Multiple Physiological Signal-Based Human Stress Identification Using Non-Linear Classifiers , 2013 .

[83]  R. Edelberg,et al.  Scoring criteria for response latency and habituation in electrodermal research: a critique. , 1985, Psychophysiology.

[84]  Yang Chen,et al.  Pairwise comparison matrix in multiple criteria decision making , 2016 .

[85]  Scott Janssen The Determinants of Reaction Times: Influence of Stimulus Intensity , 2015 .

[86]  Bernd Resch,et al.  Urban Emotions and Cycling Experience – Enriching Traffic Planning for Cyclists with Human Sensor Data , 2016 .

[87]  Cem Ersoy,et al.  Continuous Stress Detection Using Wearable Sensors in Real Life: Algorithmic Programming Contest Case Study , 2019, Sensors.

[88]  Antonio Artés-Rodríguez,et al.  Individual performance calibration using physiological stress signals , 2015, ArXiv.