This article is more than 5 years old. Information may no longer be current.

Sponsorship of medical home evaluations did not influence findings

The quantitative results of independent and nonindependent medical home evaluations were not statistically different; however, the tone of the conclusions was more favorable in nonindependent evaluations, according to a new study published in JAMA Internal Medicine.

“Published evaluations of medical home interventions have reported mixed results,” Anna Oh, MSN, MPH, RN, from the University of California, San Francisco, and colleagues wrote. “Some evaluators have been employed or funded by the sponsors of the interventions that they evaluated, while others were independent. Relationships between evaluators and sponsors of medical home interventions might influence evaluation designs and decisions about whether to publish evaluation findings. If such influence exists, evaluators employed or funded by intervention sponsors might be more likely to publish favorable findings (or suppress unfavorable findings), compared with evaluators who are independent — thus potentially biasing the medical home literature toward favorable effect estimates.”

Oh and colleagues searched several databases for keywords such as, “medical home,” “health-care home,” “comprehensive primary care,” “advanced primary care” and “patient aligned care team,” and identified 45 evaluations of 23 unique medical home interventions. These evaluations compared patient experience, technical quality, cost or utilization of care of participating practices with nonparticipating practices.

They masked sponsor information, then computed the percentage of articles that reported that the medical home intervention was associated with statistically favorable quantitative results, such as higher quality and/or lower costs. The tone of each article’s conclusion was rated as favorable, neutral or unfavorable. Articles deemed as favorable indicated better patient experience or quality and lower costs or utilization; those deemed as neutral indicated limited, modest or potential improvements in patient experience or quality and no associations with costs or utilization; and those with unfavorable conclusions indicated worse patient experience or quality, higher costs and increased utilization. After unmasking sponsor information, the researchers categorized evaluations as “not independent” or “independent.” “Not independent” evaluations included those in which the intervention sponsor funded the evaluation or employed one or more authors. All other evaluations were considered “independent.”

The researchers found that most evaluations were nonindependent (32 of 45). Patient experience was less frequently reported by independent evaluations than nonindependent evaluations (0% vs. 31%), but technical quality measures was more likely to reported (85% vs. 50%). There were no statistically significant differences in the proportion of favorable quantitative results between independent and nonindependent evaluations. Nonindependent evaluations were more likely than independent evaluations to have conclusions with a favorable tone (78% vs. 46%). Results did not vary when analyzing nonindependence due to evaluation funding or evaluator employment separately.

“We found no statistically significant differences between the quantitative results of independent and nonindependent medical home evaluations,” Oh and colleagues concluded. “However, the tone of narrative conclusions was more likely to be positive when evaluations were nonindependent. Such narrative conclusions matter because medical home evaluations frequently include multiple quantitative measures, allowing authors leeway in crafting a takeaway message when findings are mixed.”

“We recommend that policy makers and other stakeholders focus on the quantitative findings of medical home evaluations and consider evaluator independence when reading authors’ descriptions of medical home interventions’ overall effectiveness,” they added. – by Alaina Tedesco

Disclosure: Oh reports no relevant financial disclosures. Please see study for complete list of all other authors’ relevant financial disclosures.