NCCN’s off-label cancer treatment recommendations ‘constitute a problem’
Click Here to Manage Email Alerts
Many novel, costly and toxic cancer therapies continue to be broadly recommended for off-label use by the National Comprehensive Cancer Network based upon no evidence, low-levels of evidence or evidence not clearly cited in guidance documents, according to Vinay K. Prasad, MD, MPH.
In a 2018 retrospective study published in British Medical Journal, Prasad and colleagues found NCCN guidelines frequently recommend that newer branded drugs be used in settings beyond their FDA-approved indications. Razelle Kurzrock, MD, director of the Center for Personalized Cancer Therapy and Rare Tumor Clinic at University of California San Diego Moores Cancer Center, and colleagues reanalyzed the data and concluded that cancer therapies recommended for off-label use by NCCN guidelines are, in fact, supported by robust evidence, a finding in contradiction to that of Prasad and colleagues.
“These recommendations often mandate reimbursement by federal payers and, together, these facts constitute a problem — it does not serve patients to downplay public policy concerns or invent alternative hierarchies of evidence to justify the use of these agents,” Prasad, hematologist-oncologist and assistant professor of medicine at Oregon Health & Science University, and also a HemOnc Today Next Gen Innovator, told HemOnc Today.
HemOnc Today invited Prasad to address the conclusions Kurzrock and colleagues reached in their analysis, published in Annals of Oncology, and to discuss what practicing oncologists should take away from all of the data. To read an interview with Kurzrock about her group’s work, click here.
Question: Can you summarize your findings?
Answer: The NCCN compendium is one of several that mandate reimbursement for cancer drugs for off-label purposes, and the NCCN recommendations often require Medicare and commercial payers to reimburse for these off-label uses. In 2016, my colleagues and I studied how often the NCCN extrapolates beyond FDA approvals. We examined 47 contemporary anticancer therapies that were approved within the preceding 5 years, remained branded and were incredibly costly. We found that the FDA had approved these 47 drugs for 69 purposes. However, in addition to these 69 purposes, we found that the NCCN had recommended the drugs for 44 additional recommendations or extrapolations. In other words, we found that the NCCN often recommends recently FDA-approved drugs beyond their FDA approvals..
We additionally examined the level of evidence provided by the NCCN in its guidelines’ documentation. We found that 36% of the time, the NCCN did not provide any cited evidence to support the recommendation in the guidance document, and a randomized phase 3 trial was cited to support the recommendation only 16% of the time. The majority of extrapolations were either unjustified or justified by small, uncontrolled studies.
Finally, in our original publication, we examined these extrapolations 20 months after the original analysis and found that just six out of the 44 additional recommendations/extrapolations received formal FDA approval during that time.
Based upon the well-accepted and long-standing hierarchy of evidence that places randomized trials at the pinnacle of evidence-based medicine and uncontrolled observational studies beneath, we found that the NCCN frequently recommends beyond FDA approvals, and that these recommendations are often based on weak evidence. This has profound implications for patients and payers.
Q: What was your reaction to the findings by Kurzrock and colleagues?
A: We were surprised to find that employees of the NCCN, in conjunction with academic authors and contributors to the NCCN guidelines (in the paper by Kurzrock and colleagues), reached the opposite conclusion as that of our paper published in BMJ. Although the authors conceded that the NCCN often recommends drugs for additional purposes beyond those for which they were approved, the authors reach the bizarre conclusion that “the strength of evidence cited by the NCCN supporting such recommendations is robust.” The reason we were surprised by their conclusion is that Kurzrock and colleagues generally agree that the predominant basis of evidence for these extrapolations is uncontrolled, nonrandomized studies. Such studies generally are not considered robust levels of evidence in the hierarchy of evidence-based medicine. We have long seen uncontrolled, nonrandomized studies resulting in misleading causal inferences regarding the efficacy of therapy, and thus we were surprised the authors chose to spin their results as “robust.”
Q: Although both papers ultimately come to different conclusions, what about your results is similar?
A: Kurzrock and colleagues concede that the NCCN often makes recommendations beyond FDA drug approvals. They concede that those recommendations mandate payers, such as CMS and commercial payers, to pay for those off-label recommendations. They do not dispute the fact that many panelists of the NCCN guidelines have frequent and high levels of financial conflict of interest. The investigators agree that only a minority of additional recommendations led to FDA approval. They also agree that when evidence is provided for extrapolations, it generally is nonrandomized.
Q: What are the major differences between your analysis and that of Kurzrock and colleagues, and how might they have led to different conclusions?
A: Where we disagree with one another is where nonrandomized evidence falls in the hierarchy of medical evidence. We use the hierarchy of evidence that was conceived by David Sacket, MD, FRSC, physician and pioneer in medicine, in 1989 that has been used in countless publications and throughout the medical literature. Kurzrock and colleagues are using an alternative hierarchy of medical evidence of which we are unfamiliar and that has never been published or documented in the literature.
In a number of instances, we found that the NCCN failed to provide cited evidence in the guidance documents. However, Kurzrock and colleagues — after consultation with the authors of the guidelines and looking through an additional document, the transparency document, which is not part of the guidelines documents — were able to locate additional studies. These generally were uncontrolled, case reports or small sample size studies that used surrogate endpoints. This might lead to a different conclusion in the sense that fewer extrapolations are supported by no provided evidence, whereas a greater fraction are supported by uncontrolled interventional studies using surrogate endpoints. However, it should not yield the conclusion that these studies constitute “robust” evidence.
There are additional differences in the analysis of Kurzrock and colleagues, including several errors in reading and interpreting the expansive NCCN guidelines documents, as well as some facts that are provided in a misleading manner. As one example, the researchers claim that the recommendation of ixazomib (Ninlaro, Takeda) with dexamethasone was supported by a randomized trial. But, the trial they cite is a 71-person randomized trial where both arms received the investigational agent, which is not a trial capable of establishing efficacy over alternatives. There are a number of other instances, which we have detailed elsewhere in a lengthy rebuttal of this paper that appears on my personal website.
Q: What should practicing oncologists take away from all the published data?
A: Two studies largely document similar findings of uncontrolled, small sample size, surrogate endpoint studies used to justify the off-label use of costly toxic novel anticancer therapies — one used the well-established hierarchy of medical evidence and concludes that the evidence is generally weak and the other used an alternative and hitherto unpublished hierarchy of medical evidence and concludes that it is “robust.” The analysis that spins the finding in a positive light includes authors of the NCCN, as well as authors of the individual guidelines. Practicing oncologists can read both analyses and decide which analysis is correct. I encourage them to take a look at my lengthy rebuttal that appears on my personal website — www.vinayakkprasad.com/papers — to detail differences between the two analyses. – by Jennifer Southall
References:
Kurzrock R, et al. Ann Oncol. 2019;doi:10.1093/annonc/mdz232.
Wagner J, et al. BMJ. 2018;doi:10.1136/bmj.k668.
For more information:
Vinay Prasad, MD, MPH, can be reached at Oregon Health and Science University, 3181 SW Sam Jackson Park Road, Portland, OR 97239; email: prasad@ohsu.edu.
Disclosure: Prasad reports no relevant financial disclosures.