Accuracy of Optical Heart Rate Sensing Technology in Wearable Fitness Trackers for Young and Older Adults: Validation and Comparison Study

Background Wearable fitness trackers are devices that can record and enhance physical activity among users. Recently, photoplethysmography (PPG) devices that use optical heart rate sensors to detect heart rate in real time have become popular and help in monitoring and controlling exercise intensity. Although the benefits of using optical heart rate monitors have been highlighted through studies, the accuracy of the readouts these commercial devices generate has not been widely assessed for different age groups, especially for the East Asian population with Fitzpatrick skin type III or IV. Objective This study aimed to examine the accuracy of 2 wearable fitness trackers with PPG to monitor heart rate in real time during moderate exercise in young and older adults. Methods A total of 20 young adults and 20 older adults were recruited for this study. All participants were asked to undergo a series of sedentary and moderate physical activities using indoor aerobic exercise equipment. In this study, the Polar H7 chest-strapped heart rate monitor was used as the criterion measure in 2 fitness trackers, namely Xiaomi Mi Band 2 and Garmin Vivosmart HR+. The real-time, second-by-second heart rate data obtained from both devices were recorded using the broadcast heart rate mode. To critically analyze the results, multiple statistical parameters including the mean absolute percentage error (MAPE), Lin concordance correlation coefficient (CCC), intraclass correlation coefficient, the Pearson product moment correlation coefficient, and the Bland-Altman coefficient were determined to examine the performances of the devices. Results Both test devices exhibited acceptable overall accuracy as heart rate sensors based on several statistical tests. Notably, the MAPE values were below 10% (the designated threshold) in both devices (GarminYoung=3.77%; GarminSenior=4.73%; XiaomiYoung=7.69%; and XiaomiSenior=6.04%). The scores for reliability test of CCC for Garmin were 0.92 (Young) and 0.80 (Senior), whereas those for Xiaomi were 0.76 (Young) and 0.73 (Senior). However, the results obtained using the Bland-Altman analysis indicated that both test optical devices underestimated the average heart rate. More importantly, the study documented some unexpected outlier readings reported by these devices when used on certain participants. Conclusions The study reveals that commonly used optical heart rate sensors, such as the ones used herein, generally produce accurate heart rate readings irrespective of the age of the user. However, users should avoid relying entirely on these readings to indicate exercise intensities, as these devices have a tendency to produce erroneous, extreme readings, which might misinterpret the real-time exercise intensity. Future studies should therefore emphasize the occurrence rate of such errors, as this will likely benefit the development of improved models of heart rate sensors.

[1]  Tomás Ward,et al.  Artifact Removal in Physiological Signals—Practices and Possibilities , 2012, IEEE Transactions on Information Technology in Biomedicine.

[2]  W. Thompson,et al.  WORLDWIDE SURVEY OF FITNESS TRENDS FOR 2020 , 2019, ACSM'S Health & Fitness Journal.

[3]  Hyung-Soon Park,et al.  Comparison of elliptical training, stationary cycling, treadmill walking and overground walking. , 2011, Gait & posture.

[4]  D. Hollander,et al.  Validity of Wearable Activity Monitors during Cycling and Resistance Exercise , 2017, Medicine and science in sports and exercise.

[5]  K. Kelly,et al.  Prevention and Treatment of Skin Aging , 2006, Annals of the New York Academy of Sciences.

[6]  Michael Böhm,et al.  Resting heart rate: risk indicator and emerging risk factor in cardiovascular disease. , 2015, The American journal of medicine.

[7]  R. Furberg,et al.  Systematic review of the validity and reliability of consumer-wearable activity trackers , 2015, International Journal of Behavioral Nutrition and Physical Activity.

[8]  Michael Lang,et al.  Beyond Fitbit: A Critical Appraisal of Optical Heart Rate Monitoring Wearables and Apps, Their Current Limitations and Legal Implications , 2017 .

[9]  Chuen Seng Tan,et al.  Fitbit Charge HR Wireless Heart Rate Monitor: Validation Study Conducted Under Free-Living Conditions , 2017, JMIR mHealth and uHealth.

[10]  Giuseppe Coppola,et al.  Wrist-worn optical and chest strap heart rate comparison in a heterogeneous sample of healthy individuals and in coronary artery disease patients , 2018, BMC Sports Science, Medicine and Rehabilitation.

[11]  Dermot Phelan,et al.  Accuracy of Wrist-Worn Heart Rate Monitors , 2017, JAMA cardiology.

[12]  Mohammad B. Shamsollahi,et al.  Heart Rate monitoring during physical exercise using wrist-type photoplethysmographic (PPG) signals , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[13]  Terry K Koo,et al.  A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. , 2016, Journal Chiropractic Medicine.

[14]  Ahmad Alwosheel,et al.  Heart rate variability estimation in photoplethysmography signals using Bayesian learning approach , 2016, Healthcare technology letters.

[15]  A. Lim Aesthetics and Cosmetic Surgery for Darker Skin Types , 2008 .

[16]  Scott W Cheatham,et al.  Concurrent validity of resting pulse-rate measurements: a comparison of 2 smartphone applications, the polar H7 belt monitor, and a pulse oximeter with bluetooth. , 2015, Journal of sport rehabilitation.

[17]  Zohn Rosen,et al.  Validation of photoplethysmography as a method to detect heart rate during rest and exercise , 2015, Journal of medical engineering & technology.

[18]  Mohanasankar Sivaprakasam,et al.  Design, development and clinical validation of a wrist-based optical heart rate monitor , 2016, 2016 IEEE International Symposium on Medical Measurements and Applications (MeMeA).

[19]  B. Franklin,et al.  Promoting and prescribing exercise for the elderly. , 2002, American family physician.

[20]  Hyun-Sung An,et al.  How accurate are the wrist-based heart rate monitors during walking and running activities? Are they accurate enough? , 2016, BMJ Open Sport & Exercise Medicine.

[21]  Mark W. Newman,et al.  When fitness trackers don't 'fit': end-user difficulties in the assessment of personal tracking device accuracy , 2015, UbiComp.

[22]  T. Parker,et al.  Congruent Accuracy of Wrist-worn Activity Trackers during Controlled and Free-living Conditions , 2018 .

[23]  J. T. Lemmer,et al.  Acsm's Guidelines for Exercise Testing and Prescription, Sixth Edition , 2001 .

[24]  R. Laukkanen,et al.  Heart rate monitors: state of the art. , 1998, Journal of sports sciences.

[25]  Jakub Parák,et al.  Evaluation of wearable consumer heart rate monitors based on photopletysmography , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[26]  Michael A. Rupp,et al.  Describing the User Experience of Wearable Fitness Technology through Online Product Reviews , 2016 .

[27]  Walter R. Thompson WORLDWIDE SURVEY OF FITNESS TRENDS FOR 2017 , 2016 .

[28]  U. Wisløff,et al.  Accuracy of Heart Rate Watches: Implications for Weight Management , 2016, PloS one.

[29]  H A Lindberg,et al.  Heart rate as a prognostic factor for coronary heart disease and mortality: findings in three Chicago epidemiologic studies. , 1980, American journal of epidemiology.

[30]  K. Kaufman,et al.  Validity of using tri-axial accelerometers to measure human movement - Part II: Step counts at a wide range of gait velocities. , 2014, Medical engineering & physics.

[31]  Chia-Cheng Chen,et al.  Assessing agreement with intraclass correlation coefficient and concordance correlation coefficient for data with repeated measures , 2013, Comput. Stat. Data Anal..

[32]  Alexander J. Rothman,et al.  Older Adults’ Experiences Using a Commercially Available Monitor to Self-Track Their Physical Activity , 2016, JMIR mHealth and uHealth.

[33]  J. Finsterer,et al.  CNS-disease affecting the heart: Brain–heart disorders , 2014, Journal of the Neurological Sciences.

[34]  M Benjamin Nelson,et al.  Validity of Consumer-Based Physical Activity Monitors for Specific Activity Types. , 2016, Medicine and science in sports and exercise.

[35]  JENNIFER A. BUNN,et al.  Current State of Commercial Wearable Technology in Physical Activity Monitoring 2015–2017 , 2018, International journal of exercise science.

[36]  Tsipi Heart,et al.  Older adults: Are they ready to adopt health-related ICT? , 2013, Int. J. Medical Informatics.

[37]  J. Vagedes,et al.  How accurate is pulse rate variability as an estimate of heart rate variability? A review on studies comparing photoplethysmographic technology with an electrocardiogram. , 2013, International journal of cardiology.

[38]  Thea J. M. Kooiman,et al.  Reliability and validity of ten consumer activity trackers , 2015, BMC Sports Science, Medicine and Rehabilitation.

[39]  P. Laguna,et al.  Photoplethysmography pulse rate variability as a surrogate measurement of heart rate variability during non-stationary conditions , 2010, Physiological measurement.

[40]  Diane L Damiano,et al.  Comparison of elliptical training, stationary cycling, treadmill walking and overground walking. Electromyographic patterns. , 2011, Gait & posture.

[41]  V. Temple,et al.  Efficacy of a peer-guided exercise programme for adolescents with intellectual disability. , 2012, Journal of applied research in intellectual disabilities : JARID.

[42]  J. Takács,et al.  Validation of the Fitbit One activity monitor device during treadmill walking. , 2014, Journal of science and medicine in sport.

[43]  Gabriele B. Papini,et al.  Methodological Shortcomings of Wrist-Worn Heart Rate Monitors Validations , 2018, Journal of medical Internet research.

[44]  K. Robinson,et al.  The effectiveness of a peer‐mentored older adult fitness program on perceived physical, mental, and social function , 2009, Journal of the American Academy of Nurse Practitioners.

[45]  Huiman X. Barnhart,et al.  Comparison of ICC and CCC for assessing agreement for data without and with replications , 2008, Comput. Stat. Data Anal..

[46]  Mary Beth Rosson,et al.  Use and Adoption Challenges of Wearable Activity Trackers , 2015 .

[47]  Adrian Tarniceriu,et al.  Evaluation of accuracy and reliability of PulseOn optical heart rate monitoring device , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[48]  Thea J. M. Kooiman,et al.  Reliability and Validity of Ten Consumer Activity Trackers Depend on Walking Speed , 2017, Medicine and science in sports and exercise.

[49]  Parmit K. Chilana,et al.  Acceptance of Commercially Available Wearable Activity Trackers Among Adults Aged Over 50 and With Chronic Illness: A Mixed-Methods Evaluation , 2016, JMIR mHealth and uHealth.

[50]  Christopher M. Schlick,et al.  Activity Tracker and Elderly , 2015, 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing.

[51]  Virginia Pensabene,et al.  Assessment of the Fitbit Charge 2 for monitoring heart rate , 2018, PloS one.

[52]  Walter R. Thompson,et al.  WORLDWIDE SURVEY OF FITNESS TRENDS FOR 2017 , 2010, ACSM'S Health & Fitness Journal.

[53]  Adolfo Muñoz,et al.  The Usefulness of Activity Trackers in Elderly with Reduced Mobility: A Case Study , 2013, MedInfo.

[54]  C. Ramsay,et al.  A photoelectric plethysmograph for the measurement of cutaneous blood flow. , 1974, Physics in medicine and biology.

[55]  John Allen Photoplethysmography and its application in clinical physiological measurement , 2007, Physiological measurement.

[56]  Kathryn Sandberg,et al.  How consumer physical activity monitors could transform human physiology research. , 2017, American journal of physiology. Regulatory, integrative and comparative physiology.

[57]  Partners,et al.  ARPANSA – Australian Radiation Protection and Nuclear Safety Agency , 2015 .

[58]  Patrick Wahl,et al.  Criterion-Validity of Commercially Available Physical Activity Tracker to Estimate Step Count, Covered Distance and Energy Expenditure during Sports Conditions , 2017, Front. Physiol..

[59]  Hoda Javadikasgari,et al.  Variable Accuracy of Wearable Heart Rate Monitors during Aerobic Exercise , 2017, Medicine and science in sports and exercise.

[60]  E. Dooley,et al.  Estimating Accuracy at Exercise Intensities: A Comparative Study of Self-Monitoring Heart Rate and Physical Activity Wearable Devices , 2017, JMIR mHealth and uHealth.

[61]  Mohamed Ismail Nounou,et al.  Are Currently Available Wearable Devices for Activity Tracking and Heart Rate Monitoring Accurate, Precise, and Medically Beneficial? , 2015, Healthcare informatics research.

[62]  G Y Zou,et al.  Sample size formulas for estimating intraclass correlation coefficients with precision and assurance , 2012, Statistics in medicine.

[63]  Walter R. Thompson,et al.  WORLDWIDE SURVEY OF FITNESS TRENDS FOR 2016: 10th Anniversary Edition , 2015 .