Measuring the fitness of fitness trackers

Data collected by fitness trackers could play an important role in improving the health and well-being of the individuals who wear them. Many insurance companies even offer monetary rewards to participants who meet certain steps or calorie goals. However, in order for it to be useful, the collected data must be accurate and also reflect real-world performance. While previous studies have compared step counts data in controlled laboratory environments for limited periods of time, few studies have been done to measure performance over longer periods of time, while the subject does real-world activities. There are also few direct comparisons of a range of health indicators on different fitness tracking devices. In this study, we compared step counts, calories burned, and miles travelled data collected by three pairs of fitness trackers over a 14-day time period in free-living conditions. Our work indicates that the number of steps reported by different devices worn simultaneously could vary as much as 26%. At the same time, the variations seen in distance travelled, based on the step count, followed the same trends. Little correlation was found between the number of calories burned and the variations seen in the step count across multiple devices. Our results demonstrate that the reporting of health indicators, such as calories burned and miles travelled, are heavily dependent on the device itself, as well as the manufacturer's proprietary algorithm to calculate or infer such data. As a result, it is difficult to use such measurements as an accurate predictor of health outcomes, or to develop a consistent criteria to rate the performance of such devices in head-to-head comparisons.

[1]  Thea J. M. Kooiman,et al.  Reliability and validity of ten consumer activity trackers , 2015, BMC Sports Science, Medicine and Rehabilitation.

[2]  Raymond C Browning,et al.  A comparison of energy expenditure estimation of several physical activity monitors. , 2013, Medicine and science in sports and exercise.

[3]  K. Kaufman,et al.  Validity of using tri-axial accelerometers to measure human movement - Part II: Step counts at a wide range of gait velocities. , 2014, Medical engineering & physics.

[4]  D. Warburton,et al.  Health benefits of physical activity: the evidence , 2006, Canadian Medical Association Journal.

[5]  J. Takács,et al.  Validation of the Fitbit One activity monitor device during treadmill walking. , 2014, Journal of science and medicine in sport.

[6]  Tim Olds,et al.  The validity of consumer-level, activity monitors in healthy adults worn in free-living conditions: a cross-sectional study , 2015, International Journal of Behavioral Nutrition and Physical Activity.

[7]  Mark A Tully,et al.  The validation of Fibit Zip™ physical activity monitor as a measure of free-living physical activity , 2014, BMC Research Notes.

[8]  Remko R. Lengton,et al.  Measuring steps with the Fitbit activity tracker: an inter-device reliability study , 2015, Journal of medical engineering & technology.

[9]  S. Blair,et al.  Effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy , 2012, BDJ.

[10]  Shaw Bronner,et al.  Comparison of steps and energy expenditure assessment in adults of Fitbit Tracker and Ultra to the Actical and indirect calorimetry , 2013, Journal of medical engineering & technology.

[11]  K. Volpp,et al.  Accuracy of smartphone applications and wearable devices for tracking physical activity data. , 2015, JAMA.