This article is more than 5 years old. Information may no longer be current.

Fitness trackers inconsistently collect data

Wrist-worn activity trackers accurately measure heart rate, but not energy expenditure, according to findings recently published in the Journal of Personalized Medicine.

“Prior studies of wrist-worn devices have focused on earlier stage devices, or have focused exclusively on [heart rate] or estimation of [energy expenditure]. Some have made comparisons among devices without reference to the FDA-approved gold standard. None proposed an error model or framework for device validation. In response to this need, we formulated an approach to the public dissemination of validation data for consumer devices,” Anna Shcherbina, graduate student, department of medicine, Stanford University, and colleagues wrote.

“People are basing life decisions on the data provided by these devices," Euan Ashley, DPhil, FRCP, professor of cardiovascular medicine, of genetics and of biomedical data science, Stanford University, said in a press release. “But consumer devices aren’t held to the same standards as medical-grade devices, and it’s hard for doctors to know what to make of heart-rate data and other data from a patient’s wearable device.”

To gather more data on the subject, researchers evaluated the Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn and the Samsung Gear S2 on a group of 60 volunteers of varying age, fitness level, height, skin tone and weight as they sat, cycled, walked or ran on a laboratory treadmill with differing levels of intensity. Data were collected using the instructions that came with the devices or by using an Application Programming Interface.

Shcherbina and colleagues found that the lowest error in measuring heart rate was observed for the cycle ergometer task (1.8%; 95% CI, 0.9–2.7), and the highest error was observed for the walking task (5.5%; 95% CI, 3.9–7.1). All of the devices achieved a median error below 5% for heart rate on the cycle ergometer task except for the Samsung Gear S2, which achieved a median error rate of 5.1% (95% CI, 2.3–7.9). For the walking task, the Apple Watch achieved a median error rate of 2.5% (95% CI, 1.1–3.9); the PulseOn achieved a median error rate of 4.9% (95% CI, 1.4–8.6); and the Microsoft Band achieved a median error rate of 5.6% (95% CI, 4.9–6.3). The remaining four devices had median error between 6.5% and 8.8%. Across devices and modes of activities, the Apple Watch achieved the lowest error in heart rate, 2% (95% CI, 1.2–2.8), while the Samsung Gear S2 had the highest heart rate error, 6.8% (95% CI, 4.6–9).

Researchers also found that when it came to energy expenditure, median error rates across tasks varied from 27.4% (95% CI, 24–30.8) for the Fitbit Surge to 92.6% (95% CI, 87.5–97.7) for the PulseOn. The lowest relative error for this data set across the devices were achieved for the walking (31.8%; 95% CI, 28.6–35), and running (31%; 95% CI, 28–34) tasks, and the highest on the sitting tasks (52.4%; 95% CI, 48.9–57).

“Individuals and practitioners should be aware of the strengths and limitations of consumer devices that measure heart rate and estimate energy expenditure,” Shcherbina and colleagues wrote. “We encourage transparency from device companies and consistent release of validation data to facilitate the integration of such data into clinical care.”

Some of these findings are in contrast to an earlier study, which suggested that wrist-worn activity trackers yield unreliable heart rate data. Another study indicated that these devices fail to improve health outcomes, even when paired with incentives. – by Janel Miller

Disclosure: The researchers report no relevant financial disclosures.