The Accuracy of Smart Devices for Measuring Physical Activity in Daily Life: Validation Study

Background Wearables for monitoring physical activity (PA) are increasingly popular. These devices are not only used by consumers to monitor their own levels of PA but also by researchers to track the behavior of large samples. Consequently, it is important to explore how accurately PA can be tracked via these devices. Objectives The aim of this study was, therefore, to investigate convergent validity of 3 Android Wear smartwatches—Polar M600 (Polar Electro Oy, Kempele, Finland), Huawei Watch (Huawei Technologies Co, Ltd, Shenzhen, Guangdong, China), Asus Zenwatch3 (AsusTek Computer Inc, Taipei, Taiwan)—and Fitbit Charge with an ActiGraph accelerometer for measuring steps and moderate to vigorous physical activity (MVPA) on both a day level and 15-min level. Methods A free-living protocol was used in which 36 adults engaged in usual daily activities over 2 days while wearing 2 different wearables on the nondominant wrist and an ActiGraph GT3X+ accelerometer on the hip. Validity was evaluated on both levels by comparing each wearable with the ActiGraph GT3X+ accelerometer using correlations and Bland-Altman plots in IBM SPSS 24.0. Results On a day level, all devices showed strong correlations (Spearman r=.757-.892) and good agreement (interclass correlation coefficient, ICC=.695-.885) for measuring steps, whereas moderate correlations (Spearman r=.557-.577) and low agreement (ICC=.377-.660) for measuring MVPA. Bland-Altman revealed a systematic overestimation of the wearables for measuring steps but a variation between over- and undercounting of MVPA. On a 15-min level, all devices showed strong correlations (Spearman r=.752-.917) and good agreement (ICC=.792-.887) for measuring steps, whereas weak correlations (Spearman r=.116-.208) and low agreement (ICC=.461-.577) for measuring MVPA. Bland-Altman revealed a systematic overestimation of the wearables for steps but under- or overestimation for MVPA depending on the device. Conclusions In sum, all 4 consumer-level devices can be considered accurate step counters in free-living conditions. This study, however, provides evidence of systematic bias for all devices in measurement of MVPA. The results on a 15-min level also indicate that these devices are not sufficiently accurate to provide correct real-time feedback.

[1]  Catrine Tudor-Locke,et al.  Comparison of step outputs for waist and wrist accelerometer attachment sites. , 2015, Medicine and science in sports and exercise.

[2]  L. Piwek,et al.  Can Programming Frameworks Bring Smartphones into the Mainstream of Psychological Science? , 2016, Front. Psychol..

[3]  I-Min Lee,et al.  Physical activity and public health: updated recommendation for adults from the American College of Sports Medicine and the American Heart Association. , 2007, Medicine and science in sports and exercise.

[4]  Patty Freedson,et al.  Calibration of accelerometer output for children. , 2005, Medicine and science in sports and exercise.

[5]  Genevieve Fridlund Dunton,et al.  Feasibility and Performance Test of a Real-Time Sensor-Informed Context-Sensitive Ecological Momentary Assessment to Capture Physical Activity , 2016, Journal of medical Internet research.

[6]  D. French,et al.  What are the most effective intervention techniques for changing physical activity self-efficacy and physical activity behaviour--and are they the same? , 2011, Health education research.

[7]  Nuala M. Byrne,et al.  Assessment of Physical Activity and Energy Expenditure: An Overview of Objective Measures , 2014, Front. Nutr..

[8]  Thea J. M. Kooiman,et al.  Reliability and validity of ten consumer activity trackers , 2015, BMC Sports Science, Medicine and Rehabilitation.

[9]  R. Furberg,et al.  Systematic review of the validity and reliability of consumer-wearable activity trackers , 2015, International Journal of Behavioral Nutrition and Physical Activity.

[10]  Chuen Seng Tan,et al.  Fitbit Charge HR Wireless Heart Rate Monitor: Validation Study Conducted Under Free-Living Conditions , 2017, JMIR mHealth and uHealth.

[11]  Margaret Schneider,et al.  Validation of the Fitbit Zip for monitoring physical activity among free-living adolescents , 2016, BMC Research Notes.

[12]  G Plasqui,et al.  Daily physical activity assessment with accelerometers: new insights and validation studies , 2013, Obesity reviews : an official journal of the International Association for the Study of Obesity.

[13]  Walking and measurement. , 2008, Medicine and science in sports and exercise.

[14]  Catrine Tudor-Locke,et al.  Comparison of pedometer and accelerometer accuracy under controlled conditions. , 2003, Medicine and science in sports and exercise.

[15]  D. French,et al.  What is the best way to change self-efficacy to promote lifestyle and recreational physical activity? A systematic review with meta-analysis. , 2010, British journal of health psychology.

[16]  Blaine Reeder,et al.  Health at hand: A systematic review of smart watch uses for health and wellness , 2016, J. Biomed. Informatics.

[17]  W. Brown,et al.  Estimating Physical Activity and Sedentary Behavior in a Free-Living Context: A Pragmatic Comparison of Consumer-Based Activity Trackers and ActiGraph Accelerometry , 2016, Journal of medical Internet research.

[18]  L. Andersen,et al.  Accelerometer-measured daily physical activity related to aerobic fitness in children and adolescents , 2011, Journal of sports sciences.

[19]  Paul D Loprinzi,et al.  The Relationship of Actigraph Accelerometer Cut-Points for Estimating Physical Activity With Selected Health Outcomes , 2012, Research quarterly for exercise and sport.

[20]  Marsha Dowda,et al.  Equating accelerometer estimates of moderate-to-vigorous physical activity: in search of the Rosetta Stone. , 2011, Journal of science and medicine in sport.

[21]  S. Blair,et al.  Effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy , 2012, BDJ.

[22]  C. Tudor-Locke,et al.  How Many Steps/Day Are Enough? , 2004, Sports medicine.

[23]  Pedro C Hallal,et al.  Worldwide prevalence of physical inactivity and its association with human development index in 76 countries. , 2011, Preventive medicine.

[24]  Lora Giangregorio,et al.  Behavior Change Techniques Present in Wearable Activity Trackers: A Critical Analysis , 2016, JMIR mHealth and uHealth.

[25]  C. Caspersen,et al.  Physical activity, exercise, and physical fitness: definitions and distinctions for health-related research. , 1985, Public health reports.

[26]  P S Freedson,et al.  Calibration of the Computer Science and Applications, Inc. accelerometer. , 1998, Medicine and science in sports and exercise.

[27]  Alvaro Gauterin,et al.  Comparison of wrist-worn Fitbit Flex and waist-worn ActiGraph for measuring steps in free-living adults , 2017, PloS one.

[28]  Andrea M. Comptour,et al.  Validity and reliability of Fitbit activity monitors compared to ActiGraph GT3X+ with female adults in a free-living environment. , 2017, Journal of science and medicine in sport.

[29]  Phillip Tuso.  Strategies to Increase Physical Activity. , 2015, The Permanente journal.

[30]  D. Warburton,et al.  A systematic review of the evidence for Canada's Physical Activity Guidelines for Adults , 2010, The international journal of behavioral nutrition and physical activity.

[31]  Gunnar Hartvigsen,et al.  Using Fitness Trackers and Smartwatches to Measure Physical Activity in Research: Analysis of Consumer Wrist-Worn Wearables , 2018, Journal of medical Internet research.

[32]  G. Cardon,et al.  Actigraph GT3X: Validation and Determination of Physical Activity Intensity Cut Points , 2013, International Journal of Sports Medicine.

[33]  C. Dolea,et al.  World Health Organization , 1949, International Organization.

[34]  Minsoo Kang,et al.  Convergent validity of the International Physical Activity Questionnaire (IPAQ): meta-analysis , 2012, Public Health Nutrition.

[35]  M. Petre,et al.  Wearables , 2014 .

[36]  Klaus Gebel,et al.  Validity and Reliability of Fitbit Flex for Step Count, Moderate to Vigorous Physical Activity and Activity Energy Expenditure , 2016, PloS one.

[37]  Patrick Wahl,et al.  Criterion-Validity of Commercially Available Physical Activity Tracker to Estimate Step Count, Covered Distance and Energy Expenditure during Sports Conditions , 2017, Front. Physiol..

[38]  C. Tudor-Locke,et al.  Motion sensor accuracy under controlled and free-living conditions. , 2004, Medicine and science in sports and exercise.

[39]  Bernard C. K. Choi,et al.  Daily step goal of 10,000 steps: a literature review. , 2007, Clinical and investigative medicine. Medecine clinique et experimentale.

[40]  Blaine A. Price,et al.  Wearables: has the age of smartwatches finally arrived? , 2015, Commun. ACM.

[41]  J. Takács,et al.  Validation of the Fitbit One activity monitor device during treadmill walking. , 2014, Journal of science and medicine in sport.

[42]  R. Eston,et al.  Influence of speed and step frequency during walking and running on motion sensor output. , 2007, Medicine and science in sports and exercise.

[43]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[44]  B. Ainsworth,et al.  International physical activity questionnaire: 12-country reliability and validity. , 2003, Medicine and science in sports and exercise.

[45]  Inbal Nahum-Shani,et al.  Visualization of time-series sensor data to inform the design of just-in-time adaptive stress interventions , 2015, UbiComp.

[46]  Gregory M Dominick,et al.  Physical Activity Assessment Between Consumer- and Research-Grade Accelerometers: A Comparative Study in Free-Living Conditions , 2016, JMIR mHealth and uHealth.

[47]  Tim Olds,et al.  The validity of consumer-level, activity monitors in healthy adults worn in free-living conditions: a cross-sectional study , 2015, International Journal of Behavioral Nutrition and Physical Activity.

[48]  Mark A Tully,et al.  The validation of Fibit Zip™ physical activity monitor as a measure of free-living physical activity , 2014, BMC Research Notes.