Study: ‘Spin’ a consistent problem in sample of 2006 RCTs

Boutron I. JAMA. 2010;303(20):2058-2064.

In their sampling of randomized controlled trials with statistically nonsignificant results for the primary outcome, European researchers found that reporting and interpretation of findings were often inconsistent with results.

To identify the nature and frequency of “spin” in published reports, Isabelle Boutron, MD, PhD, and colleagues performed a Medline search through PubMed for randomized controlled trials (RCTs) published in December 2006. They used the Cochrane Highly Sensitive Search Strategy to identify appropriate trials. The researchers included only parallel-group RCTs with a clearly identified primary outcome showing statistically nonsignificant results.

Of the 616 RCTs published during that time, 72 met this trial’s eligibility criteria. Two readers used a pretested, standardized data abstraction form to review each paper. The form was developed in a previous pilot test.

The results showed that in 13 articles, the title contained spin (18%; 95% CI, 10.0%-28.9%). The researchers found spin in the “Results and Conclusions” section of 27 abstracts (37.5%; CI, 26.4%-49.7%) and 42 reports (58.3%; 95% CI, 46.1%-69.8%); the conclusions of 17 focused only on treatment effectiveness (23.6%; 95% CI, 14.4%-35.1%).

They also identified spin in the main text in the following areas:

Results: 21 articles (29.2%; 95% CI, 19%–41.1%);
Discussion: 31 articles (43.1%; 95% CI, 31.4%–55.3%); and
Conclusions: 36 articles (50%; 95% CI, 38%–62%).

“More than 40% of the reports had spin in at least two of these sections in the main text,” Boutron and colleagues wrote.

The paper presents some interesting findings for those of us who write as well as edit for medical journals. It has always been difficult to persuade journals to accept reports of neutral, as opposed to clearly negative, findings for publication. This has been the case even when the results come from well-planned, double-blind and carefully conducted trials. Inevitably, this may lead authors to “spin” their results and conclusions to say something “new” to justify publication.

The other less obvious pressure to “spin” may come from drug companies and implant manufacturers, who may have invested considerable time and financial support in the clinical trial. For this reason alone, it is essential that authors provide clear information on any conflict of interest if they, or their institution, have benefited in this way. The other requirement is that journal reviewers have the necessary statistical knowledge to recognize any errors in the presentation and interpretation of results.

My own opinion would be that there is a clear need to report non-significant or negative findings from properly conducted trials to guide and inform clinical practice. This should not be difficult in our age of digital technology by the use of more online reporting, perhaps in a new electronic Journal of Negative Clinical Trials.

David L. Hamblen, PhD, FRCS
Former Consulting Editor
Orthopaedics Today Europe