Accuracy of consumer-level and research-grade activity trackers in ambulatory settings in older adults

Wrist-worn activity trackers have experienced a tremendous growth lately and studies on the accuracy of mainstream trackers used by older adults are needed. This study explores the performance of six trackers (Fitbit Charge2, Garmin VivoSmart HR+, Philips Health Watch, Withings Pulse Ox, ActiGraph GT9X-BT, Omron HJ-72OITC) for estimating: steps, travelled distance, and heart-rate measurements for a cohort of older adults. Eighteen older adults completed a structured protocol involving walking tasks, simulated household activities, and sedentary activities. Less standardized activities were also included, such as: dusting, using a walking aid, or playing cards, in order to simulate real-life scenarios. Wrist-mounted and chest/waist-mounted devices were used. Gold-standards included treadmill, ECG-based chest strap, direct observation or video recording according to the activity and parameter. Every tracker showed a decreasing accuracy with slower walking speed, which resulted in a significant step under-counting. A large mean absolute percentage error (MAPE) was found for every monitor at slower walking speeds with the lowest reported MAPE at 2 km/h being 7.78%, increasing to 20.88% at 1.5 km/h, and 44.53% at 1 km/h. During household activities, the MAPE climbing up/down-stairs ranged from 8.38–19.3% and 10.06–19.01% (dominant and non-dominant arm), respectively. Waist-worn devices showed a more uniform performance. However, unstructured activities (e.g. dusting, playing cards), and using a walking aid represent a challenge for all wrist-worn trackers as evidenced by large MAPE (> 57.66% for dusting, > 67.32% when using a walking aid). Poor performance in travelled distance estimation was also evident during walking at low speeds and climbing up/down-stairs (MAPE > 71.44% and > 48.3%, respectively). Regarding heart-rate measurement, there was no significant difference (p-values > 0.05) in accuracy between trackers placed on the dominant or non-dominant arm. Concordant with existing literature, while the mean error was limited (between -3.57 bpm and 4.21 bpm), a single heart-rate measurement could be underestimated up to 30 beats-per-minute. This study showed a number of limitations of consumer-level wrist-based activity trackers for older adults. Therefore caution is required when used, in healthcare or in research settings, to measure activity in older adults.

[1]  Lisa Gualtieri,et al.  Can a Free Wearable Activity Tracker Change Behavior? The Impact of Trackers on Adults in a Physician-Led Wellness Group , 2016, JMIR research protocols.

[2]  R. Furberg,et al.  Systematic review of the validity and reliability of consumer-wearable activity trackers , 2015, International Journal of Behavioral Nutrition and Physical Activity.

[3]  Giuseppe Coppola,et al.  Wrist-worn optical and chest strap heart rate comparison in a heterogeneous sample of healthy individuals and in coronary artery disease patients , 2018, BMC Sports Science, Medicine and Rehabilitation.

[4]  Matilde Grøndahl Petersen,et al.  Validity and Reliability Evaluation of Four Commercial Activity Trackers' Step Counting Performance. , 2018, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[5]  Chuen Seng Tan,et al.  Fitbit Charge HR Wireless Heart Rate Monitor: Validation Study Conducted Under Free-Living Conditions , 2017, JMIR mHealth and uHealth.

[6]  G. Welk,et al.  How valid are wearable physical activity trackers for measuring steps? , 2017, European journal of sport science.

[7]  Merrill D. Funk,et al.  Validity of wrist-worn consumer products to measure heart rate and energy expenditure , 2018, Digital health.

[8]  Helena M. Mentis,et al.  The bit doesn't fit: Evaluation of a commercial activity-tracker at slower walking speeds. , 2018, Gait & posture.

[9]  Thea J. M. Kooiman,et al.  Reliability and validity of ten consumer activity trackers , 2015, BMC Sports Science, Medicine and Rehabilitation.

[10]  Matthew P Buman,et al.  Twenty-four Hours of Sleep, Sedentary Behavior, and Physical Activity with Nine Wearable Devices. , 2016, Medicine and science in sports and exercise.

[11]  Leonard A Kaminsky,et al.  Comparison of four Fitbit and Jawbone activity monitors with a research-grade ActiGraph accelerometer for estimating physical activity and energy expenditure , 2017, British Journal of Sports Medicine.

[12]  Scott E Crouter,et al.  Accuracy of Consumer Monitors for Estimating Energy Expenditure and Activity Type , 2017, Medicine and science in sports and exercise.

[13]  Cara Bailey Fausset,et al.  Older Adults’ Acceptance of Activity Trackers , 2017, Journal of applied gerontology : the official journal of the Southern Gerontological Society.

[14]  Junkai Xu,et al.  Validity of FitBit, Jawbone UP, Nike+ and other wearable devices for level and stair walking. , 2016, Gait & posture.

[15]  Stéphane Hallé,et al.  Validation of a Wearable Biometric System’s Ability to Monitor Heart Rate in Two Different Climate Conditions under Variable Physical Activities , 2017 .

[16]  I. Mcalister,et al.  Wrist-Worn Physical Activity Trackers Tend To Underestimate Steps During Walking , 2017 .

[17]  Brendan O'Flynn,et al.  A Review of Activity Trackers for Senior Citizens: Research Perspectives, Commercial Landscape and the Role of the Insurance Industry , 2017, Sensors.

[18]  Lora Giangregorio,et al.  Behavior Change Techniques Present in Wearable Activity Trackers: A Critical Analysis , 2016, JMIR mHealth and uHealth.

[19]  C. Dillon,et al.  Step-Count Accuracy of 3 Motion Sensors for Older and Frail Medical Inpatients. , 2017, Archives of physical medicine and rehabilitation.

[20]  Jennifer A Bunn,et al.  Assessment of step accuracy using the Consumer Technology Association standard , 2018, Journal of sports sciences.

[21]  Brendan O'Flynn,et al.  Experimental Validation of the Tyndall Portable Lower-limb Analysis System with Wearable Inertial Sensors , 2016 .

[22]  JENNIFER A. BUNN,et al.  Current State of Commercial Wearable Technology in Physical Activity Monitoring 2015–2017 , 2018, International journal of exercise science.

[23]  Miao-Ju Hsu,et al.  Accuracy of Wristband Activity Monitors during Ambulation and Activities. , 2016, Medicine and science in sports and exercise.

[24]  Subhas Chandra Mukhopadhyay Wearable Electronics Sensors : For Safe and Healthy Living , 2015 .

[25]  Gunnar Hartvigsen,et al.  Using Fitness Trackers and Smartwatches to Measure Physical Activity in Research: Analysis of Consumer Wrist-Worn Wearables , 2018, Journal of medical Internet research.

[26]  Virginia Pensabene,et al.  Assessment of the Fitbit Charge 2 for monitoring heart rate , 2018, PloS one.

[27]  Kenneth N. Brown,et al.  Human activity recognition for emergency first responders via body-worn inertial sensors , 2017, 2017 IEEE 14th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[28]  E. Savage,et al.  Perceptions, Expectations, and Informal Supports Influence Exercise Activity in Frail Older Adults , 2015 .

[29]  Iztok Fister,et al.  Sensors and Functionalities of Non-Invasive Wrist-Wearable Devices: A Review , 2018, Sensors.

[30]  S. Insana,et al.  Caloric expenditure assessment among older adults: Criterion validity of a novel accelerometry device , 2014, Journal of health psychology.

[31]  Subhas Chandra Mukhopadhyay,et al.  Wearable Sensors for Human Activity Monitoring: A Review , 2015, IEEE Sensors Journal.

[32]  Clayon B Hamilton,et al.  Accuracy of Fitbit Devices: Systematic Review and Narrative Syntheses of Quantitative Data , 2018, JMIR mHealth and uHealth.

[33]  Thea J. M. Kooiman,et al.  Reliability and Validity of Ten Consumer Activity Trackers Depend on Walking Speed , 2017, Medicine and science in sports and exercise.

[34]  Brendan O'Flynn,et al.  Novel Smart Glove Technology as a Biomechanical Monitoring Tool , 2015 .

[35]  Matthew P. Buman,et al.  Validation of Consumer-Based Hip and Wrist Activity Monitors in Older Adults With Varied Ambulatory Abilities , 2017, The journals of gerontology. Series A, Biological sciences and medical sciences.

[36]  Hyun-Sung An,et al.  How accurate are the wrist-based heart rate monitors during walking and running activities? Are they accurate enough? , 2016, BMJ Open Sport & Exercise Medicine.

[37]  C. Boissier,et al.  Performance analysis of walking of 10,000 regular users of a connected activity tracker. , 2018, Journal de medecine vasculaire.

[38]  Liezel Cilliers,et al.  Mobile and wearable technologies in healthcare for the ageing population , 2018, Comput. Methods Programs Biomed..

[39]  L. Phillips,et al.  A Comparison of Accelerometer Accuracy in Older Adults. , 2015, Research in gerontological nursing.

[40]  Adrian Bauman,et al.  The validity and reliability of consumer-grade activity trackers in older, community-dwelling adults: A systematic review. , 2018, Maturitas.

[41]  Andrea Ancillao,et al.  Indirect Measurement of Ground Reaction Forces and Moments by Means of Wearable Inertial Sensors: A Systematic Review , 2018, Sensors.

[42]  Alvaro Gauterin,et al.  Comparison of wrist-worn Fitbit Flex and waist-worn ActiGraph for measuring steps in free-living adults , 2017, PloS one.

[43]  Junqing Xie,et al.  Evaluating the Validity of Current Mainstream Wearable Devices in Fitness Tracking Under Various Physical Activities: Comparative Study , 2018, JMIR mHealth and uHealth.

[44]  P. S. St John,et al.  Comparison of ActiGraph GT3X+ and StepWatch Step Count Accuracy in Geriatric Rehabilitation Patients. , 2016, Journal of aging and physical activity.

[45]  Kaigang Li,et al.  Heart rate measures from the Apple Watch, Fitbit Charge HR 2, and electrocardiogram across different exercise intensities , 2019, Journal of sports sciences.

[46]  C. Thøgersen-Ntoumani,et al.  Reliability and validity of two fitness tracker devices in the laboratory and home environment for older community-dwelling people , 2018, BMC Geriatrics.

[47]  A. Bock,et al.  Innovation and Leadership , 2015 .

[48]  Hassan Ghasemzadeh,et al.  How Accurate Is Your Activity Tracker? A Comparative Study of Step Counts in Low-Intensity Physical Activities , 2017, JMIR mHealth and uHealth.

[49]  Young Soo Suh,et al.  Walking Parameters Estimation Based on a Wrist-Mounted Inertial Sensor for a Walker User , 2017, IEEE Sensors Journal.

[50]  M Benjamin Nelson,et al.  Validity of Consumer-Based Physical Activity Monitors for Specific Activity Types. , 2016, Medicine and science in sports and exercise.

[51]  Brendan O'Flynn,et al.  A Comprehensive Comparison of Commercial Wrist- Worn Trackers in a Young Cohort in a Lab- Environment , 2018, 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops).

[52]  Patrick Wahl,et al.  Criterion-Validity of Commercially Available Physical Activity Tracker to Estimate Step Count, Covered Distance and Energy Expenditure during Sports Conditions , 2017, Front. Physiol..