Quantitative Research
Determining the Validity and Accuracy of Multiple Activity-Tracking Devices in Controlled and Free-Walking Conditions
American Journal of Health Promotion 1-8 ª The Author(s) 2018 Reprints and permission: sagepub.com/journalsPermissions.nav DOI: 10.1177/0890117118763273 journals.sagepub.com/home/ahp
Daniel V. Gaz, MS1, Thomas M. Rieck, MA1, Nolan W. Peterson, MS1, Jennifer A. Ferguson, MS1, Darrell R. Schroeder, MS1,2, Heather A. Dunfee, PT, DPT1, Jill M. Henderzahs-Mason, PT, DPT, OCS1, and Philip T. Hagen, MD1,3
Abstract Purpose: Clinicians and fitness professionals are increasingly recommending the use of activity trackers. This study compares commercially available activity tracking devices for step and distance accuracy in common exercise settings. Design: Cross sectional. Setting: Rochester, Minnesota. Participants: Thirty-two men (n ¼ 10) and women (n ¼ 22) participated in the study. Measures: Researchers manually counted steps and measured distance for all trials, while participants wore 6 activity tracking devices that measured steps and distance. Analysis: We computed the difference between the number of steps measured by the device and the actual number of steps recorded by the observers, as well as the distance displayed by the device and the actual distance measured. Results: The analyses showed that both the device and walking trials affected the accuracy of the results (steps or distance, P < .001). Hip-based devices were more accurate and consistent for measuring step count. No significant differences were found among devices or locations for the distance measured. Conclusions: Hip-based activity tracking devices varied in accuracy but performed better than their wrist-based counterparts for step accuracy. Distance measurements for both types of devices were more consistent but lacked accuracy. Keywords accelerometer activity tracking, pedometer
Purpose According to the Centers for Disease Control and Prevention, a dramatic increase in obesity has occurred in the United States over the past 20 years, and 35% of US adults are obese.1 A recent study showed “that high levels of sedentary time and low levels of moderate to vigorous physical activity are strong and independent predictors of early death from any cause.”2(pp.1–4) Activity tracking devices have been shown to increase daily physical activity3, but the reliability and validity of numerous commercially available devices remain unclear. Currently, only the Fitbit has been validated in previous research4-8 as an accurate pedometer in a variety of conditions, including free-living9 and treadmill walking.8 Other accelerometers have been researched,10-12 but none has proven both accurate and readily available to consumers. An independent analysis of fitness technology forecasted that more than 56 million wearable fitness devices would ship in 2017.13 With so many activity-tracking devices on the
market, it is important for clinicians and fitness professionals to understand the accuracy of the different devices. A recent survey14 found that health and fitness professionals are most often asked the following 2 questions about these wearable activity devices: “Which device should I purchase?” and “How accurate are the devices?” Therefore, the purpose of this
1
Department of Internal Medicine, Mayo Clinic Healthy Living Program, Rochester, MN, USA 2 Division of Biomedical Statistics and Informatics, Mayo Clinic Healthy Living Program, Rochester, MN, USA 3 Division of General Internal Medicine, Mayo Clinic Healthy Living Program, Rochester, MN, USA Corresponding Author: Philip T. Hagen, Division of General Internal Medicine, Mayo Clinic, 200 First St SW, Rochester, MN 55905, USA. Email:
[email protected]
2
American Journal of Health Promotion XX(X)
research study was to determine which commercially available activity-tracking devices accurately track steps and distance in controlled and free-walking conditions.
Hip-based Fitbit One. The One is a small, lightweight, triaxial, accelerometry-based device that measures steps taken, floors climbed, distance traveled, calories expended, and sleep quality.
Method
Fitbit Zip. The zip is a triaxial accelerometer that can measure steps taken, distance traveled, and calories expended. The Zip monitor is smaller than that of the One and uses a watch battery for an extended battery life.
Design This study had a cross-sectional design.
Sample Thirty-two healthy men (n ¼ 10) and women (n ¼ 22) volunteered to participate in the study. Participants had no known orthopedic limitations that would prevent them from completing 30 minutes of continuous ambulation. They had no absolute contradictions to physical activity, as defined by the American College of Sports Medicine.15 The participants were employees of our institution, recruited through word of mouth and through a workplace e-mail distribution list. This study was approved by the institutional review board. The procedures and purpose of the study were explained to participants before they signed an informed consent document. Each participant also completed a Physical Activity Readiness Questionnaire.16 Each participant wore 2 hip-based activity trackers (Fitbit One, Fitbit Zip [Fitbit, Inc], San Francisco, CA) and 4 wristbased activity trackers (Apple Watch [Apple Inc], Cupertino, CA, Fitbit Charge HR [Fibit, Inc], Garmin Vivofit2 [Garmin Ltd], Olathe, KS, and Jawbone UP2 [Jawbone], San Francisco, CA) that measured steps taken and distance covered. In the treadmill trial, participants moved at predetermined speeds on a treadmill for a set amount of time. In the free-walking trial, participants walked a predetermined 1.00 mile (1.61 km) route at a self-selected pace. Imperial units were chosen because they were the default displays on all 6 devices.
Measures Health Information Participants wore lightweight clothing and no shoes to have their height and weight recorded via a scale and stadiometer (Seca 769, Seca GmbH & Co. KG, Hamburg, Germany). Height was recorded to the nearest 0.1 cm, and weight was recorded to the nearest 0.1 kg. Blood pressure was assessed to the nearest mm Hg using an electronic, automatic blood pressure cuff (BPM200, BpTRU). The dominant hand was assessed by asking the participant which hand they used for writing. A licensed physical therapist conducted a brief walking-gait analysis over approximately 100 ft of indoor, carpeted space to determine any gait deviations or abnormalities that would alter hip or wrist-based accelerometry readings.
Instruments The following descriptions were derived from each manufacturer’s product information:
Wrist-based Fitbit Charge HR. The charge HR is a wrist-based triaxial accelerometer that measures steps taken, distance traveled, floors climbed, calories expended, and sleep quality. The device also has a built-in optical heart rate sensor that measures the user’s heart rate. Apple Watch. The Apple Watch is a wrist-based smart watch with a triaxial accelerometer and gyroscope that can measure steps taken, distance traveled, calories expended, and time spent standing. This device can also measure the user’s heart rate through an optical heart rate sensor or pair with a Bluetooth chest-strap heart rate monitor. Garmin Vivofit2. The Vivofit2 is a wrist-based, 3-dimensional, accelerometer-based device that measures steps taken, distance traveled, calories expended, and sleep quality. When paired with a Garmin heart rate chest strap, the device can also measure the user’s heart rate. Jawbone UP2. The UP2 is a wrist-based, 3-dimensional, accelerometer-based device that measures steps taken, distance traveled, calories expended, and sleep quality. The UP2 also assesses sleep and physical activity patterns throughout the day.
Device Setup For all trials, participants simultaneously wore the 6 activity trackers. Two activity trackers were worn at the participant’s hip: the Fitbit One and Fitbit Zip. Four activity trackers were worn on the participant’s wrist: Fitbit Charge HR, Apple Watch, Garmin Vivofit2, and Jawbone UP2. All devices were updated with the participant’s age, sex, height, dominant hand, and weight and were worn according to the manufacturer’s recommendations. All device firmware and connected iPad/iPhone software were updated to the latest available version as of December 24, 2015. Step counts for all studies were tallied to the nearest step by 2 members of the study team, each using a manual counter (Sportline 385 Mechanical Tally Counter [EB Sport Group]). Distance for the free-walking study (condition 2) was recorded using a manual distance measuring wheel (Roadrunner RR182 [Keson]) by measuring the walking route 5 times and then taking the mean distance for an ending point.
Gaz et al.
Randomization A randomization schedule was created which for each participant assigned a device location for each wrist-based and each hip-based device, as well as the order of which condition was conducted first (free walking or treadmill). The randomization was performed using blocks of 4 ensuring that in each block of 4 participants: Each of the 4 wrist-based devices was tested once in each wrist location. Each of the 2 hip-based devices was tested twice in each hip location. Two of the 4 participants in the given block began with the treadmill, and 2 began with the free-walking condition. Two devices of each type were available for testing. For this reason, 2 complete sets of devices were created with each set including a single device of each type. In order to ensure that locations were balanced for each set of devices, a given set of devices was assigned to each block of 4 participants.
Treadmill Trial The purpose of the treadmill trials was to compare the step count and distance on the 6 activity-tracking devices using controlled, indoor treadmill walking, with manually counted steps and treadmill-reported distance. Each participant was fitted with the 6 activity trackers in a randomized order, as specified above, on their dominant wrist and hip. Participants were instructed to wait 10 seconds for the treadmill (Trackmaster TMX425CP, Full Vison, Inc, Newton, KS) to stabilize speed, then to step onto the moving belt and release their grip on the treadmill handles (so as not to interfere with normal armswing motions). Each participant walked on the treadmill for 5 minutes at 3 speeds: 1.5, 2.5, and 3.5 mph, all at a 0% grade (40.2, 67.1 m/min, and 93.9 m/min, respectively). After each 5minute exercise period, the participants were asked to step off, straddle the belt, and stand motionless to allow the researcher to record step and distance data from the activity-tracking devices. Treadmill distances of 0.11, 0.14, and 0.27 miles were based on the calibrated treadmill’s electronic display output after each trial.
Free-Walking Trial The purpose of the free-walking trial was to compare step count and distance for a 1.00 mile (1.61 km) free-walking trial of the activity-tracking devices and to compare these results with manually counted steps and distance. Participants were instructed to walk at their own self-selected pace throughout the predefined route, starting with their dominant leg. Participants were told to obey culturally accepted pedestrian safety guidelines and were guided along the route by a trained member of the research team. Upon completion of the route, participants were asked to stand motionless so that the
3 researcher could record step and distance data from the activity-tracking devices.
Analysis Each participant was assessed in 4 walking conditions in the 2 walking trials (free walking and using a treadmill at 1.5, 2.5, and 3.5 mph). In all conditions, the difference between the number of steps measured by the device and the actual number of steps recorded by the observers as well as the distance displayed by the device and the actual distance measured (device to actual) was computed. To take into account the repeated measures study design, analyses were performed using mixed linear models (PROC MIXED, SAS Institute Inc, Cary, NC), with an unstructured covariance matrix. For this analysis, the dependent variable was the difference in the number of steps and distance (device to actual), and the explanatory variables were condition, device, and condition-by-device interaction. Supplemental analyses were performed separately for each condition. For each device, the mean difference from actual was compared to zero using the absolute least-squares mean estimate. In all cases, 2-tailed P values
