Randomized trials not always the best study option

Randomized trials can deliver succinct, unbiased outcomes, yet they are an inappropriate design choice for some studies.

Once a physician decides to conduct a study or trial, a decision on which study type to use must follow. They can choose from a hierarchy of studies from evidence-based medicine, which in most cases places randomized trials in the top grouping, cohort studies in the middle, and case-control studies at the bottom.

The trials at the top of the hierarchy reduce the potential influence of bias, making the outcome less tainted while minimizing the risk of human error. “This information will help investigators plan the best study possible, and help those reading manuscripts interpret the data,” said William M. Ricci, MD, an associate professor of orthopedic surgery at Washington University School of Medicine in St. Louis.

Ricci presented his ideas on the hierarchy of trials at the 2005 Orthopedic Trauma Association (OTA) meeting in Ottawa. He compared randomized trials to cohort studies, prospective cohort studies to retrospective cohort studies, and also discussed selection bias.

Different levels of evidence

Randomized trials set up identical patient groups, but randomize exposure to treatment while maintaining an equal distribution of all known and unknown prognostic factors. “Randomized studies, when possible, offer the best and most bias-free data. These studies provide the highest level of evidence,” Ricci said. “However, they are also usually the most difficult, labor intensive and costly studies to perform.”

In a cohort trial design, researchers follow patients over time, either prospectively or retrospectively, and the study is not randomized. Such studies can measure incidence and can also show causality, and offer the next best option when investigators cannot randomize. “Prospective cohort studies with a comparison group offer the best level of evidence in this situation,” Ricci said.

In a prospective cohort study, researchers set up a defined population, either exposed or not exposed. The hypothesis is postulated before patients exposure occurs and researchers follow patients over time. In a retrospective cohort study, researchers also set up a defined population and follow it over time, but the hypothesis follows exposure.

Sometimes researches may set out to do a prospective study, but later find that some of the patients’ exposure occurred prior to the development of a hypothesis, which would result in a retrospective study.

RCTs not always best

Randomized controlled trials (RCTs) top the hierarchy in most situations, but are not always the best option. In fact, RCTs may be out of the running when looking at or questioning etiologic research, diagnostic test evaluations, prognostic research, unintended treatment effects and intended treatment effects, Ricci said.

For example, Ricci discussed a study from The Journal of the American Medical Association where randomization was not possible, causing the potential for bias. The study looked at osteoporotic fractures in white women over the age of 65 years. Investigators put patients with low bone mineral density (BMD) into one group and those with normal BMD into another. According to Ricci, this cannot be randomized. Because patients either have low or normal BMD, they cannot be randomly assigned to one group or the other to determine fracture risk.

Regarding cohort studies, Ricci said the “challenge is to minimize other factors that determine the observed outcome-effect (besides treatment).” In order to have a successful cohort study, he suggested avoiding subjective outcome interpretation, minimizing confounding variables by choosing the appropriate cohort and reducing external factors.

Ricci believes non-randomized studies have value even though randomized studies provide more accurate and unbiased data. He presented situations in which randomization would be unethical, making non-randomized study the choice by default. For instance, bringing one group of patients to the operating room more emergently than another group would randomize results and differentiate outcomes, but it is unethical to knowingly delay patient treatment.

When performing a non-randomized study, Ricci suggests using a comparative study, preferably a prospective cohort design, rather than a retrospective cohort design. He also noted that doctors should carefully document all known prognostic factors and be sure not to rely solely on non-randomized study results.

For More Information:
Ricci WM. The prospective cohort study: Is it a better alternative than an RCT? Presented at the Orthopaedic Trauma Association 21st Annual Meeting. Oct. 19-22, 2005. Ottawa.