Unpublished clinical trial results a ‘violation of the covenant’ with participants

The goal of most clinical trials — including those that test a medication, assess a diagnostic device or evaluate a surgical technique — is to gather and share information about the efficacy and safety of an intervention.

Whether the outcome is positive or negative, researchers are bound to report the results so as to add to the existing body of knowledge and advance understanding.

Failure to report the results of clinical trials has been an ongoing problem for decades, and this inaction may not be victimless. Some experts fear nonreporting of clinical trial results poses a significant threat to the integrity of the overall evidence base.

Section 801 of the FDA Amendments Act (FDAAA 801), introduced in 2007, requires investigators of “applicable clinical trials” to report their results within 1 year of trial completion. The act was clarified and expanded in a Final Rule issued in 2017, which also provided more details on sanctions for noncompliance, including fines of over $12,000 per day as of this year.

Yet, despite the implementation and finalization of FDAAA 801, a study published earlier this year in The Lancet showed poor compliance with and enforcement of the rule. In addition to compromising the evidence base, noncompliance with FDAAA 801 can lead to research waste and redundancies in data.

Moreover, the important questions that are explored in clinical trials remain unanswered.

“People agree to participate in clinical trials based on an understanding that the risk they are assuming will be contributing to something larger — a larger knowledge base,” Nicholas J. DeVito, MPH, a DPhil candidate in the Nuffield department of primary care health science at University of Oxford, and an author of The Lancet study, told HemOnc Today. “That’s not the only reason results should be reported, but it certainly is a factor. So, by not reporting a trial, you’re essentially not honoring that commitment to your participants.”

HemOnc Today spoke with experts about the prevalence of trials that go unpublished by both industry and academic sponsors, the implications this lack of reporting has for patients with cancer, and the leadership needed to curtail this trend.

Prevalence of nonpublication

According to ClinicalTrials.gov, the statutory requirements for FDAAA 801 have been in place since Sept. 27, 2007. The clarified regulation took effect Jan. 18, 2017, with investigators required to comply beginning April 18 of that year.

“The rule that came out in 2017 was just implementing the existing 2007 law, and people were given notice of it well in advance,“ Evan Mayo-Wilson, MPA, DPhil, associate professor in the department of epidemiology and biostatistics at Indiana University School of Public Health-Bloomington, told HemOnc Today. “There was maybe a full year leading up to the enforcement date. So, even if you went by the 2007 rule rather than the law, we all had plenty of notice about this.”

Despite this notice, DeVito and colleagues found poor compliance with the new change to the rule. They found that as of September 2019, of 4,209 trials due to report results, only 1,722 (40.9%; 95% CI, 39.4-42.2) did so within the 1-year deadline, meaning nearly 60% of trials were not in compliance with the law. In total, 2,686 trials (63.8%; 95% CI, 62.4-65.3) submitted results at any time.

When looking at trends in compliance from March 2018 to September 2019, the researchers found that it did not improve with time, remaining stable at approximately 40%.

In another study that looked specifically at oncology trials prior to FDAAA 801 finalization, Chapman and colleagues analyzed all clinical studies that closed to accrual at Memorial Sloan Kettering Cancer Center between 2009 and 2013.

Results showed 469 of 809 (58%) protocols were published by November 2016. The researchers calculated that the probability of publication 7 years after finishing trial enrollment was 70.4%, meaning that approximately 30% of trials would be unpublished 7 years after closing to accrual.

Researchers reported that 30.6% of trials had closed prior to completing accrual, mainly due to poor accrual or pharmaceutical sponsor issues.

However, 29% of completed but unpublished trials had been considered not of sufficient interest by the investigators to warrant publication; many of these had negative results.

DeVito said the original FDAAA 801 rule had some elements that were ambiguous in terms of what trials were required to report and when. When the Final Rule was implemented, it was prospective, basically exempting all violations prior to 2017.

However, in February, a federal court ruled that FDAAA 801 also should require the reporting of results of trials that were completed prior to implementation of the Final Rule. The ruling was in a lawsuit brought by the Media, Freedom & Information Access Clinic at Yale Law School against HHS, the NIH and the FDA. Because the chance to appeal the lawsuit has passed, the ruling is final. In the FAQ section on ClinicalTrials.gov, the NIH has since instructed trials sponsors to report missing results “as soon as possible.”

“Hopefully robust efforts are made to reach sponsors of trials covered by the ruling who have not reported results.” DeVito said. “It’s encouraging that this requirement is now clearly in place, but whether it actually has any impact will come down to implementation and enforcement.”

‘Improved practices’ from industry

Perhaps surprisingly, DeVito and colleagues’ study showed that pharmaceutical companies have not been the worst violators of FDAAA 801.

In fact, industry sponsors appeared significantly more likely to be compliant (OR = 3.08; 95% CI, 2.52-3.77) and report trial results (OR = 1.62; 95% CI, 1.35-1.96) than nonindustry, non-U.S. government sponsors.

These data suggest pharmaceutical companies have responded to efforts to boost compliance, according to Jennifer E. Miller, PhD, assistant professor in Yale School of Medicine and director of the Good Pharma Scorecard initiative.

“There have been several initiatives focused on improving clinical trial transparency for privately sponsored trials for the pharmaceutical industry,” Miller told HemOnc Today. “They have really helped the industry clean up its act.”

The Good Pharma Scorecard, which Miller directs along with Yale colleague Joseph S. Ross, MD, and Michelle Mello, JD, PhD, of Stanford, ranks new drugs and pharmaceutical companies on their ethics, social responsibility and “patient centricity.” Since it was established in 2015, the Good Pharma Scorecard has noted consistent improvements in pharmaceutical company compliance year after year, Miller said.

“We look at all new drugs approved in a given year, find all the trials that were conducted to gain regulatory approval for that drug, then see what percentage of those trials have public results,” Miller said. “We found about half of companies with a low score will improve their practices within 30 days of getting a low score, suggesting ranking companies is an effective reform strategy. This is likely in part because what gets measured often gets done.”

Pharmaceutical companies have not always been compliant with FDAAA 801. Mayo-Wilson noted that there was an uptick in compliance after the Final Rule was implemented.

“Drug manufacturers and device manufacturers that have been doing trials of products in order to get regulatory approval sometimes had published the results of those trials to help with the marketing effort,” Mayo-Wilson said. “Before the Final Rule was implemented, there was little incentive for them to promote the results of trials that did not have positive results.”

Erick H. Turner, MD, associate professor of psychiatry at Oregon Health & Science University, has conducted studies on reporting of industry-sponsored trials with negative findings prior to the 2017 Final Rule.

In a study published in 2008 in The New England Journal of Medicine, Turner and colleagues found that of 74 FDA-registered studies evaluating antidepressants, 31% were not published.

Moreover, studies deemed by the FDA to have negative or questionable results were either not published (22 studies) or were published in a way that made them appear positive (11 studies). Only three studies with negative results were published as a failed study.

“They were taking advantage of the fact that there was no law that mandated that they had to publish these studies; it was perfectly legal. It was just ethically wrong,” he said. “Now, with the Final Rule, it’s the law and everyone should know that, but some investigators are still not putting the results online. It seems that the FDA is complicit in letting them off the hook and not collecting the fines they’re legally mandated to collect.”

Academic institutions: Delegating responsibility

On the flip side, academic institutions appear to be doing a poorer job complying with FDAAA 810.

A report published in 2019 by TranspariMED and Universities Allied for Essential Medicines North America found that 25 of the 40 universities that sponsor the most trials in the United States did not publish study results within 1 year of completion, per the 2017 Final Rule requirements. Of 450 trials completed by the 40 universities, results of 140 studies appeared to be missing from the public registry.

The report showed Columbia University had reported the lowest proportion of trials on time, at 17%, whereas University of California, San Francisco had the largest number of unreported trials, at 37% of its total.

“The problem since 2017 has been much more noticeable in academia,” Mayo-Wilson said. “Large academic medical centers run as many trials as drug companies, and people have not paid as much attention to this in academia as they have in industry.

“I think that is changing, though,” he added. “Large academic medical centers are aware that they are now at risk of losing NIH funding if they don’t follow the rules.”

Since the time of that 2019 report, the reporting figures of both Columbia University and UCSF have dramatically improved, with Columbia now at 100% and UCSF at 91.2%, according to FDAAA TrialsTracker.

In a statement issued to HemOnc Today, UCSF officials said they improved reporting compliance through a multifaceted approach.

“We created and filled a position dedicated to ClinicalTrials.gov regulatory support, fully implemented a tool built at UCLA called PROCoM Tracker (Protocol Registration and Results System Record Oversight and Compliance Management), participated as an institution in the Clinical Trials Registration and Results Reporting Taskforce and tailored outreach to clinical researchers to increase compliance,” officials said in the statement. “As part of our tailored outreach, we offer consultations to help researchers understand the requirements of FDAAA 801, clarify and help resolve PRS [Protocol Registration and Results System] review comments related to results reporting and offer personalized support and follow-up.”

Mayo-Wilson said pharmaceutical companies have the advantage of dedicated compliance departments, which ensure that clinical trials comply with the rules of the country in which they are conducted.

“Academic medical centers don’t have as many people supporting these kinds of activities. NIH and the law anticipate that the organization will have procedures in place to ensure compliance,” he said. “If I accept money from NIH as an investigator, the contract is actually not with me; it’s with my institution. However, academic medical centers and universities in general have delegated this responsibility to the individual investigator.”

Mayo-Wilson said it is not realistic to expect every individual physician conducting a small clinical trial to be familiar with the details of the Final Rule, which is 700 pages long. He said some institutions have started to recognize this and have begun to hire professional staff who specialize in compliance.

“Delegating that responsibility to the principal investigator is not a realistic way to get good compliance,” he said.

Vinay K. Prasad, MD, MPH, hematologist-oncologist and associate professor of medicine in the division of epidemiology and biostatistics at UCSF, and a HemOnc Today Next Gen Innovator, agreed that pharmaceutical companies are much more practiced at meeting regulatory requirements.

“I think that’s a general theme: when you have a hoop to jump through, industry will always do better than academics; they’re good at following rules,” he told HemOnc Today. “In this case, academic institutions have ignored their responsibility. Obviously, they are more than happy to run the trials, so why aren’t they disseminating the results? It’s because they’re making money from running the trials and they’re very happy to use this as a business model.”

Prasad added that institutions can benefit from clinical trials through marketing campaigns distinguishing them from local hospitals.

“The fact that an academic institution is running a trial is often advertised as a reason for patients to go there instead of to their community hospital,” he said. “They’re more than happy to take advantage of it as a marketing tool and collect the revenue from it, but they don’t want to actually disseminate the results. That’s a violation of the covenant researchers make with patients who participate in the studies.”

Turner believes that, in some cases, the failure on the part of academic institutions to report their results is due to lack of emphasis at the leadership level, lack of awareness or negligence.

“One would have to do a survey to determine their level of awareness. Is someone reinforcing this, telling them they need to report results?” he said. “There is probably some paperwork that they do; hopefully they’re not checking boxes thoughtlessly when applying for grants.”

Turner added that NIH is supposed to withhold future funding for academic grants from investigators who don’t comply with FDAAA 801.

“This raises the question: Is the NIH enforcing these rules any better than the FDA is?” he said.

The NIH Office of Extramural Research noted in a statement issued to HemOnc Today that “the policy took effect for all competing applications and contract proposals submitted on or after Jan. 18, 2017, and for clinical trials initiated by the NIH intramural research program on or after Jan. 18, 2017.” NIH further stated that, “at this time, most studies would not have reached their results reporting due date under NIH policy.”

If the FDA issues a notice of noncompliance under FDAAA 810, the NIH told HemOnc Today that it would “post it to ClinicalTrials.gov and take other actions as appropriate, including notifying grantees, providing 30 days for correction and withholding funds.”

NIH cited its ongoing reinforcement of internal processes, adding that it aims to help researchers report clinical trial results through education and outreach, including tutorials, workshops and assistance in navigating the process.

“We will continue these efforts and take any other actions afforded to us to preserve our investment in the biomedical research enterprise,” the statement reads.

Lost knowledge

DeVito said failure to publish clinical trial results is tantamount to obscuring the “entire picture” of the evidence in a disease or treatment area.

“If we’re looking at treating individuals or to synthesize evidence in a systematic review or meta-analysis and we don’t have all the information based on the study that was conducted, we can’t make the most informed decisions,” he said.

“Then there’s the research waste component; the value of the trial comes from the knowledge it generates,” he added. “If you don’t share that knowledge, the resources that went into conducting that trial, and any value it could generate, are essentially lost.”

Miller said unless the knowledge from a trial is made known to the scientific community, the entire ethics of the trial are called into question.

“We have to ask ourselves, why do we allow researchers to experiment on people? Why is that allowed?” she said. “Part of the ethical justification is the potential to contribute to generalizable knowledge and help advance the common good. So, when we don’t have public results, it brings into question the ethics of a trial.”

Ethics can also be compromised through clinical trial redundancy.

“There are a few examples where, particularly in early-stage research, findings on drugs that were toxic were not reported and the drugs were given to a second group of people,” Mayo-Wilson said. “Researchers were wasting people’s time and money testing medical products that don’t need to be tested. Additionally, these products that are tested redundantly tend to be the ones that don’t work. So, from an ethical perspective, we are exposing people to harm.”

This issue of complete trial transparency is especially relevant to ongoing trials testing vaccines for COVID-19 to ensure public trust of the results, according to an article published earlier this month in STAT by Miller, Ross and Mello.

The FDAAA requirements for trial sponsors to make basic trial details and results publicly accessible only apply to studies conducted at U.S. sites or for products manufactured in the U.S., with exceptions for phase 1 studies.

“These details matter for SARS-CoV-2 vaccine research because several developers are based outside the U.S. and many trials are safety studies, not interventional,” Miller and colleagues wrote.

Six of the 12 companies the furthest along in vaccine development have been evaluated by the Good Pharma Scorecard initiative, according to the article, with three having above-average transparency scores (Johnson & Johnson, Sanofi and GlaxoSmithKline), one with an average score (Merck), and two with below-average scores (Pfizer and AstraZeneca).

Earlier this month, Pfizer and BioNTech announced their mRNA-based vaccine candidate, BNT162b2, demonstrated an efficacy rate of more than 90% in the first interim efficacy analysis of an ongoing phase 3 trial.

Miller and colleagues called on industry to be transparent in the areas of registration, public results reporting, publication, and data and protocol sharing to ensure public trust in the results and the safety and efficacy of any approved vaccines.

“The U.S. public has already endured repeated missteps in the pandemic response, so the stakes involved in bringing safe, effective vaccines to market and ensuring their uptake have never been higher,” they wrote. “Transparency will help ensure this achievement is trusted.”

Leadership, enforcement

Miller, who helped spearhead improvements in compliance among pharmaceutical companies, said major changes at academic institutions need to come from their leadership.

“You need a commitment at the top from institutions,” she said. “There also needs to be policies and procedures in place to help operationalize that commitment. Further, resources and staff must be allocated for implementation of policies. Ideally, a benchmarking and tracking system should ensure everything happens, and if it doesn’t, you figure out why.”

Miller also suggested that institutions make researchers accountable for reporting trial results by making compliance a condition of hiring.

“If a researcher is going to take a new job, it could be a point of hiring to be sure that they reported results of any studies they ran that had NIH funding,” she said. “The employer could double check that before hiring the researcher. Similarly, larger companies could audit the transparency track records of smaller companies they merge with or acquire, requiring deficiencies be fixed before partnering.”

DeVito noted that academic institutions appear to get better at compliance as they perform more trials, which increases experience and knowledge.

“A main finding we highlighted is that larger organizations tend to do better, which I think further supports the idea that if you’ve done a lot of clinical trials and if you’ve sponsored a lot of clinical trials over time, you tend to do better at reporting,” he said. “Essentially, the more experience and institutional knowledge you have, the better you are at these things.”

According to many clinicians, however, the real solution to the problem would be for the FDA to enforce the fines it had stated it would impose.

Although the FDA did not respond to requests for comment from HemOnc Today, DeVito’s study included the following statement from the agency: “The agency’s goal is to achieve voluntary compliance with the law without having to resort to legal action.”

DeVito and colleagues noted that, as of the publication of their study, no fines had been issued.

In August, FDA released guidance on how such fines would be enforced, although some experts said this guidance falls short.

Mayo-Wilson said he believes it is time for the FDA to act on its stated intention to impose fines.

“FDA has never used its authority to fine a trial sponsor for failing to meet these requirements,” Mayo-Wilson said. “I think we’ve probably gotten about as far as we’re going to with shaming drug manufacturers and academic medical centers into meeting these requirements. If we want to make headway, the FDA needs to start issuing fines.”

References:

Chapman PB, et al. PLoS One. 2017;doi:10.1371/journal.pone.0184025.

DeVito NJ, et al. Lancet. 2020;doi:10.1016/S0140-6736(19)33220-9.

FDA. Civil money penalties relating to the ClinicalTrials.gov data bank: Guidance for responsible parties, submitters of certain applications and submissions to FDA, and FDA staff. Available at: www.fda.gov/media/113361/download. Accessed Nov. 2, 2020.

Miller JE, et al. Far more transparency is needed for Covid-19 vaccine trials. STAT. Nov. 5, 2020.

TranspariMED and UAEM North America. Clinical trial transparency at US universities: Compliance with U.S. law and global best practices. Available at: www.transparimed.org/reports. Accessed Nov. a, 2020.

Turner EH et al. N Engl J Med. 2008;doi:10.1056/NEJMsa065779.

For more information:

Nicholas J. DeVito, MPH, can be reached at University of Oxford, Radcliffe Observatory Quarter, Woodstock Road, Oxford OX2 6GG; email: nicholas.devito@phc.ox.ac.uk.

Evan Mayo-Wilson, MPA, DPhil, can be reached at Indiana University School of Public Health, 1025 E. 7th St., Bloomington, IN 47405; email: emayowil@iu.edu.

Jennifer E. Miller, PhD, can be reached at Yale University School of Medicine, 367 Cedar St., Harkness 4th Floor, New Haven, CT 06520; email: jennifer.e.miller@yale.edu; Twitter: @millerbioethics.

Vinay K. Prasad, MD, MPH, can be reached at University of California, San Francisco, 550 16th St., 2nd floor, San Francisco, CA 94158; Twitter: @vprasadmdmph.

Erick H. Turner, MD, can be reached at 3181 SW Sam Jackson Park Road, Portland, OR 97239; email: turnere@ohsu.edu.