Success of phase-2 trials not accurate predictor of phase-3 trial outcome

Every 10 assessable patients added to phase-2 studies increases the likelihood of a positive phase-3 study 1.18 times.

Issue: March 1, 2006

The established method for developing new antitumor agents or chemotherapeutic regimens is conducting 6 trials that test the efficacy of promising experimental treatments as they progress through phases 1, 2 and 3. Data from phase-2 trials are generally used to determine whether a more rigorous phase-3 trial is indicated.

However, relying on phase 2 is not the most prudent course according to a study published in the Journal of Clinical Oncology by Mohammad Zia, MD, of the department of medical oncology and hematology at the University of Toronto.

“Our study shows that promising results from phase 2 studies are seldom reproduced in randomized phase 3 studies,” Zia said.

Although positive results from a phase-2 study are generally cited as the basis for conducting a phase-3 trial, there are significant differences in the rules governing trial design and result assessment that make the success of a phase-2 trial a poor indicator of the success of a phase-3 trial, he explained.

Predicting phase-3 success

Zia and colleagues researched 43 phase-3 studies from July 1998 through June 2003 that compared experimental chemotherapeutic treat- ments with standard regimens and used identical chemotherapy regimens to the phase-2 studies that preceded them.

Then, they undertook an analysis of the 49 phase-2 studies, examining the number of patients enrolled, number of assessable patients, the reported response rate, and whether the study was randomized. The researchers determined the phase-3 study resulting from each phase-2 study to be either positive or negative, depending on whether the primary endpoints were reached.

The researchers investigated which factors in a phase-2 study showed correlation with the success of the phase-3 study. Zia discovered that “the majority of phase 3 studies based on preceding phase 2 studies were ‘negative.’” The only predictive factor was assessable patients. A 10-patient increase in assessable patients in phase-2 studies increased the likelihood of a positive phase-3 study 1.18 times.

Study design differences

Zia and colleagues provided several explanations for the study findings. Over a given period, fluctuations in the measurement of a variable occur, according to the regression-to-truth phenomenon. Response rates in any phase-2 study could, by chance, be “far removed from the ‘true’ response rate,” according to Zia. A phase-2 study might then be taken to a phase-3 trial, where the “true” response rate might emerge as a result of further testing. Phase-2 studies with a small number of participants intensify this effect.

Selection bias in nonrandomized phase-2 studies might also have affected their predictive qualities, according to Zia. He said that the unintentional “enrollment of high proportions of patients with good prognostic factors” might have exaggerated the studies’ promising results. In the ensuing phase-3 studies, in which patient selection is randomized, this inflationary factor was eliminated.

Additionally, phase-2 studies generally calculate response rates using only the results from assessable patients, whereas phase-3 studies report data from all enrolled patients. According to Zia, this study design discrepancy could inflate the response rate in phase-2 studies.

The last explanatory factor is a result of newer imaging techniques available when the phase-2 studies were conducted. Since phase-3 trials follow phase 2, Zia said that more advanced techniques available to the phase-3 researchers could reveal a lower number of partial response patients — classifying them as having a stable or progressive disease – than there were in the phase-2 study. Also, in phase-3 trials, there are independent reviews of images, while in phase-2 trials, external reviews were “infrequent.”

Changes for the future?

Several remedies to this problem have been proposed, according to Zia, but none have been evaluated or validated. For example, only multi-institutional phase-2 studies should be considered as a basis for phase 3, in order to eliminate the regression-to-truth phenomenon and selection bias. Also, the decision to proceed to phase 3 could be made at intervals during phase 2 based on “predictive probabilities that the experimental therapy was superior given the observed data.” – by Janet F. Mays

For more information:

Zia MI, Siu LL, Pond GR, Chen EX. Comparison of outcomes of phase II studies and subsequent randomized control studies using identical chemotherapeutic regimens. J Clin Oncol. 2005;23:6982-6991.