Automatic and Intelligent Stressor Identification Based on Photoplethysmography Analysis

A stressor is an external stimulus that an individual might consider demanding or threatening regarding his/her safety. Stressors cause the release of stress hormones and can be categorized into two categories: physiological and psychological. Wearable technology has been on the rise in the past decade, and the advancement of IoT will only further the benefits that wearable technology will bring. This work leverages the output of wearable technology to provide automatic stress and stressor identification model. In particular, this study proposes a novel algorithm that first detects instances of stress and then classifies the stressor type using photoplethysmography (PPG) data from wearable smartwatches. The time-series PPG data is transformed into 2D spatial images by first extracting the interbeat interval (IBI), and the blood volume pulse (BVP) features from the PPG, then transforming these data to 2D image data. The developed model successfully identified stress instances using IBI-BVP spatial domain images with an average accuracy of 98.10% with a convolutional neural network (CNN) and 99.18% using the average pixel intensity of these images with the extra trees classifier. It also successfully categorized stressor types into physical, cognitive, and social with an accuracy of 98.5% using the CNN and 96.45% using the extra trees classifier, outperforming state-of-the-art.

[1]  Hui Han,et al.  Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning , 2005, ICIC.

[2]  Anderson Rocha,et al.  Pixel-Level Tissue Classification for Ultrasound Images , 2016, IEEE Journal of Biomedical and Health Informatics.

[3]  P A Kyriacou,et al.  Photoplethysmography for blood volumes and oxygenation changes during intermittent vascular occlusions , 2017, Journal of Clinical Monitoring and Computing.

[4]  Prerita Kalra,et al.  Mental Stress Assessment Using PPG Signal a Deep Neural Network Approach , 2020, IETE Journal of Research.

[5]  Matjaz Gams,et al.  Monitoring stress with a wrist device using context , 2017, J. Biomed. Informatics.

[6]  J. Henry,et al.  The short-form version of the Depression Anxiety Stress Scales (DASS-21): construct validity and normative data in a large non-clinical sample. , 2005, The British journal of clinical psychology.

[7]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Muhammad Zubair,et al.  Mental Stress Assessment via Ultra-Short-Term Recordings of Photoplethysmographic Sensor , 2019, 2019 4th International Conference on Computing, Communications and Security (ICCCS).

[9]  Patrick van der Smagt,et al.  CNN-based Segmentation of Medical Imaging Data , 2017, ArXiv.

[10]  Jukka A Lipponen,et al.  A robust algorithm for heart rate variability time series artefact correction using novel beat classification , 2019, Journal of medical engineering & technology.

[11]  Matthew England,et al.  A Combined CNN and LSTM Model for Arabic Sentiment Analysis , 2018, CD-MAKE.

[12]  Cinna Soltanpur,et al.  A review on wearable photoplethysmography sensors and their potential future applications in health care , 2018, International journal of biosensors & bioelectronics.

[13]  A. Beck,et al.  Comparison of Beck Depression Inventories -IA and -II in psychiatric outpatients. , 1996, Journal of personality assessment.

[14]  Zhu Xiao,et al.  Identifying Ethnics of People through Face Recognition: A Deep CNN Approach , 2020, Sci. Program..

[15]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[16]  Anna Yokokubo,et al.  The Influence of Person-Specific Biometrics in Improving Generic Stress Predictive Models , 2019, ArXiv.

[17]  Inchan Youn,et al.  Ambulatory and Laboratory Stress Detection Based on Raw Electrocardiogram Signals Using a Convolutional Neural Network , 2019, Sensors.

[18]  Zixue Cheng,et al.  CNN for situations understanding based on sentiment analysis of twitter data , 2017 .

[19]  Tad T. Brunyé,et al.  Stress Effects on Mood, HPA Axis, and Autonomic Response: Comparison of Three Psychosocial Stress Paradigms , 2014, PloS one.

[20]  Oliver T. Wolf,et al.  The socially evaluated cold-pressor test (SECPT) for groups: Effects of repeated administration of a combined physiological and psychological stressor , 2014, Psychoneuroendocrinology.

[21]  Kristof Van Laerhoven,et al.  Introducing WESAD, a Multimodal Dataset for Wearable Stress and Affect Detection , 2018, ICMI.

[22]  Mohamed Chetouani,et al.  Person-specific behavioural features for automatic stress detection , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[23]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[24]  Pramod Bobade,et al.  Stress Detection with Machine Learning and Deep Learning using Multimodal Physiological Data , 2020, 2020 Second International Conference on Inventive Research in Computing Applications (ICIRCA).

[25]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[26]  Xiong Luo,et al.  A Novel Classification Method for a Driver’s Cognitive Stress Level by Transferring Interbeat Intervals of the ECG Signal to Pictures , 2020, Sensors.

[27]  Mika P. Tarvainen,et al.  Kubios HRV - Heart rate variability analysis software , 2014, Comput. Methods Programs Biomed..

[28]  Masatoshi Okutomi,et al.  Inter-Beat Interval Estimation from Facial Video Based on Reliability of BVP Signals , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[29]  K. Suma,et al.  Performance analysis of Machine Learning techniques for classification of stress levels using PPG signals , 2020, 2020 IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT).

[30]  L. Schwabe,et al.  Ten years of research with the Socially Evaluated Cold Pressor Test: Data from the past and guidelines for the future , 2018, Psychoneuroendocrinology.

[31]  C. Kirschbaum,et al.  The 'Trier Social Stress Test'--a tool for investigating psychobiological stress responses in a laboratory setting. , 1993, Neuropsychobiology.

[32]  M. D. De Buyzere,et al.  The Average Pixel Intensity Method and Outcome of Mitral Regurgitation in Mitral Valve Prolapse. , 2019, Journal of the American Society of Echocardiography : official publication of the American Society of Echocardiography.

[33]  Kai Xie,et al.  Face Recognition Using the SR-CNN Model , 2018, Sensors.