A standardized validity assessment protocol for physiological signals from wearable technology: Methodological underpinnings and an application to the E4 biosensor

Wearable physiological measurement devices for ambulatory research with novel sensing technology are introduced with ever increasing frequency, requiring fast, standardized, and rigorous validation of the physiological signals measured by these devices and their derived parameters. At present, there is a lack of consensus on a standardized protocol or framework with which to test the validity of this new technology, leading to the use of various (often unfit) methods. This study introduces a comprehensive validity assessment protocol for physiological signals (electrodermal activity and cardiovascular activity) and investigates the validity of the E4 wearable (an example of such a new device) on the three levels proposed by the protocol: (1) the signal level, with a cross-correlation; (2) the parameter level, with Bland–Altman plots; and (3) the event level, with the detection of physiological changes due to external stressor levels via event difference plots. The results of the protocol show that the E4 wearable is valid for heart rate, RMSSD, and SD at the parameter and event levels, and for the total amplitude of skin conductance responses at the event level when studying strong sustained stressors. These findings are in line with the prior literature and demonstrate the applicability of the protocol. The validity assessment protocol proposed in this study provides a comprehensive, standardized, and feasible method for assessment of the quality of physiological data coming from new wearable (sensor) technology aimed at ambulatory research.

[1]  Kenta Matsumura,et al.  iPhysioMeter: A new approach for measuring heart rate and normalized pulse volume using only a smartphone , 2013, Behavior research methods.

[2]  Peter J. Gianaros,et al.  Handbook of Psychophysiology: Methodology , 2007 .

[3]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .

[4]  Peter A. Dinda,et al.  UStress: Understanding college student subjective stress using wrist-based passive sensing , 2017, 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops).

[5]  K. Wilke,et al.  A short history of sweat gland biology , 2007, International journal of cosmetic science.

[6]  Ben Cowley,et al.  Feasibility of an electrodermal activity ring prototype as a research tool , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[7]  G. Willemsen,et al.  Ambulatory monitoring of the impedance cardiogram. , 1996, Psychophysiology.

[8]  Carol L. Lake,et al.  Clinical Monitoring: Practical Applications for Anesthesia and Critical Care , 2001 .

[9]  Carmen C. Y. Poon,et al.  Wearable devices and their applications in surgical robot control and p-medicine , 2016, 2016 IEEE 20th International Conference on Computer Supported Cooperative Work in Design (CSCWD).

[10]  M. Dawson,et al.  Lapses in skin conductance responding across anatomical sites: Comparison of fingers, feet, forehead, and wrist. , 2016, Psychophysiology.

[11]  E. Erdfelder,et al.  Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses , 2009, Behavior research methods.

[12]  Sylvie Charbonnier,et al.  A comparison of wearable and stationary sensors for stress detection , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[13]  P. Hancock,et al.  Noise effects on human performance: a meta-analytic synthesis. , 2011, Psychological bulletin.

[14]  Tapas Mondal,et al.  Wearable Sensors for Remote Health Monitoring , 2017, Sensors.

[15]  D. Altman,et al.  STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT , 1986, The Lancet.

[16]  Richard A. Berk,et al.  Applied Time Series Analysis for the Social Sciences , 1980 .

[17]  Noor Azina Ismail,et al.  Statistical Methods Used to Test for Agreement of Medical Instruments Measuring Continuous Variables in Method Comparison Studies: A Systematic Review , 2012, PloS one.

[18]  J. Evans.  Straightforward Statistics for the Behavioral Sciences , 1995 .

[19]  W. Boucsein.  Electrodermal activity, 2nd ed. , 2012 .

[20]  A. Hrõbjartsson,et al.  Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. , 2011, Journal of clinical epidemiology.

[21]  Alison Watkins,et al.  How honest are the signals? A protocol for validating wearable sensors , 2018, Behavior research methods.

[22]  P. Venables,et al.  Publication recommendations for electrodermal measurements. , 1981 .

[23]  G. Berntson,et al.  Handbook of Psychophysiology: Cardiovascular Psychophysiology , 2007 .

[24]  Patrick T. Brandt,et al.  Multiple Time Series Models , 2006 .

[25]  L. Chen,et al.  Reference ranges for short-term heart rate variability measures in individuals free of cardiovascular disease: The Multi-Ethnic Study of Atherosclerosis (MESA). , 2016, Journal of electrocardiology.

[26]  Willis J. Tompkins,et al.  A Real-Time QRS Detection Algorithm , 1985, IEEE Transactions on Biomedical Engineering.

[27]  David Giles,et al.  Validity of the Polar V800 heart rate monitor to measure RR intervals at rest , 2015, European Journal of Applied Physiology.

[28]  M. Benedek,et al.  A continuous measure of phasic electrodermal activity , 2010, Journal of Neuroscience Methods.

[29]  D. Streiner,et al.  Guidelines for Reporting Reliability and Agreement Studies (GRRAS). , 2011, International journal of nursing studies.

[30]  David A. Clifton,et al.  Signal-Quality Indices for the Electrocardiogram and Photoplethysmogram: Derivation and Applications to Wireless Monitoring , 2015, IEEE Journal of Biomedical and Health Informatics.

[31]  T. Reinhardt,et al.  Salivary cortisol, heart rate, electrodermal activity and subjective stress responses to the Mannheim Multicomponent Stress Test (MMST) , 2012, Psychiatry Research.

[32]  Rosalind W. Picard,et al.  Empatica E3 — A wearable wireless multi-sensor device for real-time computerized biofeedback and data acquisition , 2014 .

[33]  Nikolaos G. Bourbakis,et al.  A Survey on Wearable Sensor-Based Systems for Health Monitoring and Prognosis , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[34]  Arun Kumar Agnihotri,et al.  Determination Of Sex By Hand Dimensions , 2005 .

[35]  D. Bonett.  Sample size requirements for estimating intraclass correlations with desired precision , 2002, Statistics in medicine.

[36]  Ulrich Ebner-Priemer,et al.  Ambulatory assessment. , 2013, Annual review of clinical psychology.

[37]  Jonathan Westley Peirce,et al.  Neuroinformatics Original Research Article Generating Stimuli for Neuroscience Using Psychopy , 2022 .

[38]  W Karlen,et al.  Photoplethysmogram signal quality estimation using repeated Gaussian filters and cross-correlation , 2012, Physiological measurement.

[39]  David Nunan,et al.  Levels of agreement for RR intervals and short-term heart rate variability obtained from the Polar S810 and an alternative system , 2008, European Journal of Applied Physiology.

[40]  W. Bardwell,et al.  Effects of stress on heart rate complexity—A comparison between short-term and chronic stress , 2009, Biological Psychology.

[41]  J M Bland,et al.  Statistical methods for assessing agreement between two methods of clinical measurement , 1986 .

[42]  B. Wallin,et al.  Discharge patterns of sympathetic neurons supplying skeletal muscle and skin in man and cat. , 1983, Journal of the autonomic nervous system.

[43]  Gabriele B. Papini,et al.  Methodological Shortcomings of Wrist-Worn Heart Rate Monitors Validations , 2018, Journal of medical Internet research.

[44]  S. Wiens,et al.  Stress Recovery during Exposure to Nature Sound and Environmental Noise , 2010, International journal of environmental research and public health.

[45]  K. McGraw,et al.  Forming inferences about some intraclass correlation coefficients. , 1996 .

[46]  Wei Zhang,et al.  A Unified Framework for Street-View Panorama Stitching , 2016, Sensors.

[47]  A. Petrie,et al.  Method agreement analysis: a review of correct methodology. , 2010, Theriogenology.

[48]  C. Goodman.  Association for the Advancement of Medical Instrumentation , 1988 .

[49]  M. Dawson,et al.  The electrodermal system , 2007 .

[50]  Joris H. Janssen,et al.  Emotional sweating across the body: Comparing 16 different skin conductance measurement locations , 2012, Physiology & Behavior.

[51]  K Yamakoshi,et al.  Normalized pulse volume (NPV) derived photo-plethysmographically as a more valid measure of the finger vascular tone. , 2001, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[52]  Andy Adler,et al.  Validation of the Empatica E4 wristband , 2016, 2016 IEEE EMBS International Student Conference (ISC).

[53]  D. Giavarina.  Understanding Bland Altman analysis , 2015, Biochemia medica.

[54]  R. Edelberg,et al.  Relation of electrical properties of skin to structure and physiologic state. , 1977, The Journal of investigative dermatology.

[55]  Rosalind W. Picard,et al.  A Wearable Sensor for Unobtrusive, Long-Term Assessment of Electrodermal Activity , 2010, IEEE Transactions on Biomedical Engineering.

[56]  A. Jaggi,et al.  Preclinical experimental stress studies: protocols, assessment and comparison. , 2015, European journal of pharmacology.

[57]  C. Kirschbaum,et al.  The 'Trier Social Stress Test'--a tool for investigating psychobiological stress responses in a laboratory setting. , 1993, Neuropsychobiology.

[58]  Anne-Marie Brouwer,et al.  A Comparison of Different Electrodermal Variables in Response to an Acute Social Stressor , 2017, Symbiotic.

[59]  Rosalind W. Picard,et al.  Empatica E3 — A wearable wireless multi-sensor device for real-time computerized biofeedback and data acquisition , 2014, 2014 4th International Conference on Wireless Mobile Communication and Healthcare - Transforming Healthcare Through Innovations in Mobile and Wireless Technologies (MOBIHEALTH).

[60]  M. A. Rowe,et al.  Guide for Analysing Electrodermal Activity & Skin Conductance Responses for Psychological Experiments , 2013 .

[61]  Maarten A. Hogervorst,et al.  A new paradigm to induce mental stress: the Sing-a-Song Stress Test (SSST) , 2014, Front. Neurosci..