The Accuracy of Acquiring Heart Rate Variability from Portable Devices: A Systematic Review and Meta-Analysis

BackgroundAdvancements in wearable technology have provided practitioners and researchers with the ability to conveniently measure various health and/or fitness indices. Specifically, portable devices have been devised for convenient recordings of heart rate variability (HRV). Yet, their accuracies remain questionable.ObjectiveThe aim was to quantify the accuracy of portable devices compared to electrocardiography (ECG) for measuring a multitude of HRV metrics and to identify potential moderators of this effect.MethodsThis meta-analysis was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. Articles published before July 29, 2017 were located via four electronic databases using a combination of the terms related to HRV and validity. Separate effect sizes (ESs), defined as the absolute standardized difference between the HRV value recorded using the portable device compared to ECG, were generated for each HRV metric (ten metrics analyzed in total). A multivariate, multi-level model, incorporating random-effects assumptions, was utilized to quantify the mean ES and 95% confidence interval (CI) and explore potential moderators.ResultsTwenty-three studies yielded 301 effects and revealed that HRV measurements acquired from portable devices differed from those obtained from ECG (ES = 0.23, 95% CI 0.05–0.42), although this effect was small and highly heterogeneous (I2 = 78.6%, 95% CI 76.2–80.7). Moderator analysis revealed that HRV metric (p <0.001), position (p = 0.033), and biological sex (β = 0.45, 95% CI 0.30–0.61; p <0.001), but not portable device, modulated the degree of absolute error. Within metric, absolute error was significantly higher when expressed as standard deviation of all normal–normal (R–R) intervals (SDNN) (ES = 0.44) compared to any other metric, but was no longer significantly different after a sensitivity analysis removed outliers. Likewise, the error associated with the tilt/recovery position was significantly higher than any other position and remained significantly different without outliers in the model.ConclusionsOur results suggest that HRV measurements acquired using portable devices demonstrate a small amount of absolute error when compared to ECG. However, this small error is acceptable when considering the improved practicality and compliance of HRV measurements acquired through portable devices in the field setting. Practitioners and researchers should consider the cost–benefit along with the simplicity of the measurement when attempting to increase compliance in acquiring HRV measurements.

[1]  Wolfgang Viechtbauer,et al.  Conducting Meta-Analyses in R with the metafor Package , 2010 .

[2]  J. Weir Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. , 2005, Journal of strength and conditioning research.

[3]  R. Rosenthal Meta-analytic procedures for social research , 1984 .

[4]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[5]  K. Häkkinen,et al.  Effects of HRV-Guided vs. Predetermined Block Training on Performance, HRV and Serum Hormones , 2017, International Journal of Sports Medicine.

[6]  J. Villacastín,et al.  A novel device based on smart textile to control heart’s activity during exercise , 2014, Australasian Physical & Engineering Sciences in Medicine.

[7]  D. Rennie,et al.  Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative , 2003, Annals of Internal Medicine.

[8]  J. Hox Multilevel analysis: Techniques and applications, 2nd ed. , 2010 .

[9]  A. Booth,et al.  Literature searching for social science systematic reviews: consideration of a range of search techniques. , 2009, Health information and libraries journal.

[10]  G. Breithardt,et al.  Heart rate variability: standards of measurement, physiological interpretation and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. , 1996 .

[11]  A. Kilding,et al.  Evaluating training adaptation with heart-rate measures: a methodological comparison. , 2013, International journal of sports physiology and performance.

[12]  P. Ekkekakis,et al.  AFFECT-BASED EXERCISE PRESCRIPTION: An Idea Whose Time Has Come? , 2017 .

[13]  A. Flatt,et al.  Validity of the ithlete™ Smart Phone Application for Determining Ultra-Short-Term Heart Rate Variability , 2013, Journal of human kinetics.

[14]  Timothy J. Robinson,et al.  Multilevel Analysis: Techniques and Applications , 2002 .

[15]  A. Deslandes,et al.  Heart Rate Variability Indexes as a Marker of Chronic Adaptation in Athletes: A Systematic Review , 2015, Annals of noninvasive electrocardiology : the official journal of the International Society for Holter and Noninvasive Electrocardiology, Inc.

[16]  S. Sammito,et al.  [Options and limitations of heart rate measurement and analysis of heart rate variability by mobile devices: A systematic review]. , 2016, Herzschrittmachertherapie & Elektrophysiologie.

[17]  J P Vandenbroucke,et al.  Bias in meta-analysis detected by a simple, graphical test , 1998 .

[18]  S. Thompson,et al.  Quantifying heterogeneity in a meta‐analysis , 2002, Statistics in medicine.

[19]  Robert J Volk,et al.  Citation searches are more sensitive than keyword searches to identify studies using specific measurement instruments. , 2015, Journal of clinical epidemiology.

[20]  H. Huikuri,et al.  Time domain, geometrical and frequency domain analysis of cardiac vagal outflow: effects of various respiratory patterns. , 2001, Clinical physiology.

[21]  I. Olkin,et al.  Estimating time to conduct a meta-analysis from number of citations retrieved. , 1999, JAMA.

[22]  David Moher,et al.  Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative. , 2003, Radiology.

[23]  J. Sterne,et al.  Systematic reviews of test accuracy should search a range of databases to identify primary studies. , 2008, Journal of clinical epidemiology.

[24]  J. Vagedes,et al.  How accurate is pulse rate variability as an estimate of heart rate variability? A review on studies comparing photoplethysmographic technology with an electrocardiogram. , 2013, International journal of cardiology.

[25]  Eliete Bouskela,et al.  Heart rate variability assessment with fingertip photoplethysmography and polar RS800cx as compared with electrocardiography in obese adolescents , 2015, Blood pressure monitoring.

[26]  F. Nakamura,et al.  Agreement Between a Smartphone Pulse Sensor Application and Electrocardiography for Determining lnRMSSD. , 2016, Journal of strength and conditioning research.

[27]  D. Credeur,et al.  Telemetry‐derived heart rate variability responses to a physical stressor , 2017, Clinical physiology and functional imaging.

[28]  James A. J. Heathers,et al.  Smartphone-enabled pulse rate variability: an alternative methodology for the collection of heart rate variability in psychophysiological research. , 2013, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[29]  J. Thayer,et al.  Heart Rate Variability and Cardiac Vagal Tone in Psychophysiological Research – Recommendations for Experiment Planning, Data Analysis, and Data Reporting , 2017, Front. Psychol..

[30]  M. Buchheit,et al.  Improbable effect of carbohydrate diet on cardiac autonomic modulation during exercise , 2010, European Journal of Applied Physiology.

[31]  G. Millet,et al.  Monitoring Fatigue Status with HRV Measures in Elite Athletes: An Avenue Beyond RMSSD? , 2015, Front. Physiol..

[32]  M Kingsley,et al.  Comparison of Polar 810s and an ambulatory ECG system for RR interval measurement during progressive exercise. , 2005, International journal of sports medicine.

[33]  Jaroslaw Piskorski,et al.  Classic electrocardiogram-based and mobile technology derived approaches to heart rate variability are not equivalent. , 2018, International journal of cardiology.

[34]  A. Mellor,et al.  A comparison of two methods of heart rate variability assessment at high altitude , 2017, Clinical physiology and functional imaging.

[35]  J. Zarocostas Non-communicable diseases must have greater priority, says WHO , 2009, BMJ : British Medical Journal.

[36]  D. Nunan,et al.  Validity and reliability of short-term heart-rate variability from the Polar S810. , 2009, Medicine and science in sports and exercise.

[37]  Andrew E. Kilding,et al.  Training Adaptation and Heart Rate Variability in Elite Endurance Athletes: Opening the Door to Effective Monitoring , 2013, Sports Medicine.

[38]  Lisa Board,et al.  Validity of Telemetric-Derived Measures of Heart Rate Variability: A Systematic Review , 2016 .

[39]  Comparing cardiac output monitors and defining agreement: A systematic review and meta-analysis , 2016, Journal of the Intensive Care Society.

[40]  A. Flatt,et al.  Heart Rate Variability and Training Load Among National Collegiate Athletic Association Division 1 College Football Players Throughout Spring Camp , 2017, Journal of strength and conditioning research.

[41]  H. Nagaraja,et al.  Heart rate variability: origins, methods, and interpretive caveats. , 1997, Psychophysiology.

[42]  F. Nakamura,et al.  Initial Weekly HRV Response is Related to the Prospective Change in VO2max in Female Soccer Players , 2016, International Journal of Sports Medicine.

[43]  Walter R. Thompson WORLDWIDE SURVEY OF FITNESS TRENDS FOR 2017 , 2016 .

[44]  L. Porto,et al.  Comparison of Time‐Domain Short‐Term Heart Interval Variability Analysis Using a Wrist‐Worn Heart Rate Monitor and the Conventional Electrocardiogram , 2009, Pacing and clinical electrophysiology : PACE.

[45]  David Giles,et al.  Validity of the Polar V800 heart rate monitor to measure RR intervals at rest , 2015, European Journal of Applied Physiology.

[46]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement , 2009, BMJ : British Medical Journal.

[47]  A. Malliani,et al.  Heart rate variability. Standards of measurement, physiological interpretation, and clinical use , 1996 .

[48]  M. Buchheit Monitoring training status with HR measures: do all roads lead to Rome? , 2014, Front. Physiol..

[49]  Judith D. Singer,et al.  Using SAS PROC MIXED to Fit Multilevel Models, Hierarchical Models, and Individual Growth Models , 1998 .

[50]  Johannes B Reitsma,et al.  STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration , 2016, BMJ Open.

[51]  David Moher,et al.  Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative. , 2003, Radiology.

[52]  F. Gamelin,et al.  Validity of the Polar S810 to Measure R-R Intervals in Children , 2007, International journal of sports medicine.

[53]  A. Flatt,et al.  Evaluating Individual Training Adaptation With Smartphone-Derived Heart Rate Variability in a Collegiate Female Soccer Team , 2016, Journal of strength and conditioning research.

[54]  D. Altman,et al.  Measuring inconsistency in meta-analyses , 2003, BMJ : British Medical Journal.

[55]  Carmen Russoniello,et al.  Heart Rate Variability and Biological Age: Implications for Health and Gaming , 2013, Cyberpsychology Behav. Soc. Netw..

[56]  David Nunan,et al.  Levels of agreement for RR intervals and short-term heart rate variability obtained from the Polar S810 and an alternative system , 2008, European Journal of Applied Physiology.

[57]  Matthias Weippert,et al.  Comparison of three mobile devices for measuring R–R intervals and heart rate variability: Polar S810i, Suunto t6 and an ambulatory ECG system , 2010, European Journal of Applied Physiology.

[58]  Raquel Bailón,et al.  Validation of Heart Rate Monitor Polar RS800 for Heart Rate Variability Analysis During Exercise , 2016, Journal of strength and conditioning research.

[59]  F. Nakamura,et al.  Interpreting daily heart rate variability changes in collegiate female soccer players. , 2016, The Journal of sports medicine and physical fitness.

[60]  M. Rosenberg,et al.  THE FILE‐DRAWER PROBLEM REVISITED: A GENERAL WEIGHTED METHOD FOR CALCULATING FAIL‐SAFE NUMBERS IN META‐ANALYSIS , 2005, Evolution; international journal of organic evolution.

[61]  Barbara Canlon,et al.  Possibilities and limitations of the polar RS800 in measuring heart rate variability at rest , 2011, European Journal of Applied Physiology.

[62]  M. Altini,et al.  Comparison of Heart-Rate-Variability Recording With Smartphone Photoplethysmography, Polar H7 Chest Strap, and Electrocardiography. , 2017, International journal of sports physiology and performance.

[63]  F. Nakamura,et al.  Intraday and Interday Reliability of Ultra-Short-Term Heart Rate Variability in Rugby Union Players , 2016, Journal of strength and conditioning research.

[64]  Martin Radespiel-Tröger,et al.  Agreement of two different methods for measurement of heart rate variability , 2003, Clinical Autonomic Research.

[65]  Noor Azina Ismail,et al.  Statistical Methods Used to Test for Agreement of Medical Instruments Measuring Continuous Variables in Method Comparison Studies: A Systematic Review , 2012, PloS one.

[66]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement , 2009, BMJ.

[67]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[68]  Martin Buchheit,et al.  Day-to-Day Heart-Rate Variability Recordings in World-Champion Rowers: Appreciating Unique Athlete Characteristics. , 2017, International journal of sports physiology and performance.

[69]  Walter R. Thompson,et al.  WORLDWIDE SURVEY OF FITNESS TRENDS FOR 2018: The CREP Edition , 2017 .

[70]  Jesse A Berlin,et al.  Searching one or two databases was insufficient for meta-analysis of observational studies. , 2005, Journal of clinical epidemiology.

[71]  C. M. Pastre,et al.  Comparison of the Polar S810i monitor and the ECG for the analysis of heart rate variability in the time and frequency domains. , 2008, Brazilian journal of medical and biological research = Revista brasileira de pesquisas medicas e biologicas.

[72]  I. Böckelmann,et al.  Möglichkeiten und Einschränkungen der Herzfrequenzmessung und der Analyse der Herzfrequenzvariabilität mittels mobiler Messgeräte , 2016, Herzschrittmachertherapie + Elektrophysiologie.

[73]  C. Begg,et al.  Publication bias : a problem in interpreting medical data , 1988 .

[74]  C. M. Pastre,et al.  Comparison of Polar® RS800G3™ heart rate monitor with Polar® S810i™ and electrocardiogram to obtain the series of RR intervals and analysis of heart rate variability at rest , 2016, Clinical physiology and functional imaging.

[75]  A. Flatt,et al.  Heart rate variability stabilization in athletes: towards more convenient data acquisition , 2016, Clinical physiology and functional imaging.

[76]  J. Sterne,et al.  Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. , 2001, Journal of clinical epidemiology.

[77]  Marco Altini,et al.  HRV4Training: Large-scale longitudinal training load analysis in unconstrained free-living settings using a smartphone application , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[78]  G. A. Marcoulides,et al.  Multilevel Analysis Techniques and Applications , 2002 .

[79]  Sophia H. Quraishi,et al.  Guidelines for Reporting Articles on Psychiatry and Heart rate variability (GRAPH): recommendations to advance research communication , 2016, PloS one.

[80]  Ki H. Chon,et al.  Statistical analysis of heart rate and heart rate variability monitoring through the use of smart phone cameras , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[81]  A. Flatt,et al.  Ultra-short-term heart rate variability indexes at rest and post-exercise in athletes: evaluating the agreement with accepted recommendations. , 2014, Journal of sports science & medicine.

[82]  J. Ioannidis,et al.  Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials , 2011, BMJ : British Medical Journal.

[83]  A W Schopper,et al.  Heart rate variability: comparison among devices with different temporal resolutions. , 2005, Physiological measurement.

[84]  F. Gamelin,et al.  Validity of the polar S810 heart rate monitor to measure R-R intervals at rest. , 2006, Medicine and science in sports and exercise.