This article is more than 5 years old. Information may no longer be current.
Positive phase 3 trials more often cited, published in high-impact journals
Click Here to Manage Email Alerts
Randomized, phase 3 oncology trials with positive outcomes appeared more frequently in top scientific journals and were more frequently cited than trials with negative outcomes, according to the results of a meta-analysis.
“Phase 3 trials provide the highest level of evidence for showing the efficacy of new treatments or interventions,” Joseph M. Unger, PhD, MS, assistant member of the public health sciences division at Fred Hutchinson Cancer Research Center and the Southwest Oncology Group (SWOG) Statistical Center in Seattle, and colleagues wrote. “Given the size of the investment, negative trials — that is, those that fail to show that a new treatment is superior to standard treatment — may be incorrectly regarded as poor investments. However, negative trials are also important if they show that new treatments, which might otherwise be adopted into clinical practice, in fact do not work.”
Joseph M. Unger
Unger and colleagues surmised that a bias in favor of positive phase 3 trial results would be reflected in scientific literature. Thus, they sought to compare the scientific impact of positive vs. negative trials by assessing 94 phase 3 clinical cancer trials conducted by SWOG between 1985 and 2014 (n = 46,424).
Researchers assessed scientific impact through the publication of primary and secondary articles, the publication rate and mean impact factor. Further, they used Google Scholar to evaluate the number of articles that cited each SWOG article each year following its initial publication.
Unger and colleagues considered trial outcomes positive if they achieved a statistically significant outcome in favor of a new or experimental treatment for the protocol-specified primary endpoint. Conversely, trials achieving a statistically significant result in favor of standard therapy — or a null result with no statistically significant benefit either way — were considered negative.
Ninety-six percent of the trials (n = 90) had superiority designs, with 37% (n = 35) designed to assess survival alone and 28% (n = 26) designed to assess another time-to-event endpoint.
Twenty-six trials (28%) closed early due to poor accrual.
The researchers deemed 28% (n = 26) of completed SWOG trials positive based on protocol-specified primary endpoint analysis.
Scott F. Huntington
Among 66 trials evaluated for their primary endpoint, 100% of positive trials and 95% of negative trials (n = 38 of 40) achieved publication. The researchers did not observe an overall publication bias toward positive trials.
Overall, 273 unique articles were published during the study period, including 209 secondary articles.
The primary articles reporting results of positive trials were more often published in scientific journals with greater 2-year impact factors than negative trials (19% vs. 13%; P = .007).
A significantly greater proportion of positive trials also were published in very high-impact journals, such as JAMA, The Lancet and The New England Journal of Medicine (35% vs. 13%; P = .03).
The 66 evaluated trials were cited 24,235 times over 783 at-risk years for the primary articles. The citation rate per at-risk year was 43 for positive trials but 21 for negative trials, which represented a statistically significant difference (RR = 2; 95% CI, 1.1-3.9).
Negative trials had more cited secondary articles (12,461 vs. 6,029). However, Unger and colleagues did not observe a significant difference in the number of primary and secondary articles cited per at-risk year between positive and negative trials (55 vs. 45; RR = 1.2; 95% CI, 0.7-2.3).
Cary P. Gross
The researchers acknowledged study limitations, including the potential for incomplete data in Google Scholar and the exclusive inclusion of SWOG trials. Further, they noted that each citation does not necessarily represent a uniform level of scientific impact.
Trials that close early due to poor accrual have the least scientific impact, Unger and colleagues wrote.
“This suggests the importance of designing trials with strong support in the scientific and treatment communities,” they wrote. “Once a trial is successfully completed, the totality of its scientific impact ... promises to be substantial, regardless of whether the trial results are positive or negative.”
It is imperative for researchers and clinicians to recognize the potential value of negative trial data, Scott F. Huntington, MD, MPH, MSHP, assistant professor of hematology at Yale School of Medicine, and Cary P. Gross, MD, professor of medicine and co-director of the Robert Wood Johnson Foundation Clinical Scholars Program at Yale School of Medicine, wrote in an accompanying editorial.
Although the NCI has implemented initiatives to enhance the scientific impact of their treatment trials — such as using universal biorepositories for molecular analyses and incorporating patient-reported outcomes — negative trials are likely to persist within the NCI’s portfolio, Huntington and Gross wrote.
“This should not be considered a weakness,” they wrote. “First, clinical equipoise would suggest that trials should only be conducted when there is true uncertainty about the relative risks and benefits of different treatment arms; hence a fair proportion of studies should be expected to yield negative results. Second, negative studies can have great value, particularly if the treatment arms have similar clinical efficacy but vary in other important metrics such as cost or treatment burden.” – by Cameron Kelsall
Disclosure: The researchers report no relevant financial disclosures. Gross reports research funding from 21st Century Oncology, Johnson & Johnson, Medtronic and Pfizer. Huntington reports no relevant financial disclosures.
Click Here to Manage Email Alerts