Randomized controlled trials: Read more than just the abstract
Click Here to Manage Email Alerts
Randomized controlled trials sit atop the research triangle as the best-quality evidence available in medical research. Using the best method to randomly assign patients in a trial is critical to account for sample size and multiple covariables.
The results of these studies are often accepted as outcomes that would be true for other surgeons and their practices, influencing future decisions in patient care. Specifically, the trials can have significant impacts on updates to clinical care guidelines and FDA approvals, as well as influence payer decisions on reimbursement. Yet when the stated results are accepted without further analysis of the study design and methods, wrong conclusions may be drawn.
Devil is in the details
Recently, two randomized clinical trials were published on outcomes after implantation of the InSpace balloon (Stryker) for the management of massive rotator cuff tears. Each study was designed to be level-1 evidence, randomized and single-blinded to the patient. Both studies had large numbers of patients who were followed for 2 years. Yet the results were surprisingly different. The North American trial showed the balloon was safe and efficacious resulting in earlier improvement in patient outcomes compared with partial repair. The English study found the balloon resulted in inferior outcomes and was potentially dangerous and should not be used. How can two similar studies result in starkly different results?
As with most orthopedic research, the devil is in the details. Each study must be critically evaluated before clinical conclusions can be drawn. The study must be analyzed starting with the methods section – not just jumping to the authors’ conclusions. In doing so, one may identify specific factors which help to provide valuable clinical nuances that may impact data interpretation. What patients were included and excluded? How was randomization used to reduce selection bias among treatment groups? What was the sample size relative to the clinical question? What was the primary focus of the study and were the goals reflected in the primary outcome measures? When were outcomes measured and by whom?
As orthopedic surgeons, we recognize patient indications are one of the most important factors in determining outcomes. As an example, before we conclude that meniscectomy does not work, it would be important to separate a 30-year-old patient with normal articular surfaces and a displaced flap tear from a 70-year-old patient with a complex degenerative tear and grade III/IV articular disease in the involved compartment. Similarly, in the treatment of rotator cuff problems, factors such as preoperative range of motion, tear size, especially subscapularis involvement, and the presence of articular disease would likely have a significant impact on patient outcome. Two studies of a similar treatment may have different results because different populations were enrolled.
Sample size
A study may be powered within a population overall but may be underpowered to draw conclusions regarding specific subpopulations. We previously authored a randomized controlled trial regarding the routine use of acromioplasty during arthroscopic rotator cuff repair showing no difference in outcomes. The paper is frequently quoted in recommending against the use of acromioplasty and has been used to support negative coverage decisions regarding payment for the procedure.
As we acknowledge in the study, one primary limitation is the lack of adequate numbers to compare patients with type 3 acquired acromial morphology who have anterior osteophyte/calcification of the CA ligament. These patients may arguably have a different pathology spectrum when compared with younger patients with more acute pathology. In addition, we did not examine the rate of tendon healing between groups. We also did not follow our patients long-term to determine if retear rates were affected by an acromioplasty. As such, while the study suggests that routine use may not be beneficial, there may be a specific subpopulation in which there is a positive response.
Outcome measures
Specific outcome measures are critical to elucidate the outcome of choice. For example, minimally invasive knee and hip arthroplasty gained popularity during the last 10 years. Given that the same implants with less invasive arthroplasty are used with more traditional surgical approaches, the outcomes at 2 years postoperatively may be expected to be equivalent. However, one significant potential advantage, particularly from a patient perspective, is less pain, earlier return of function and earlier return to work.
If a study does not include specific outcome measures at the appropriate early time point, one may conclude there is no advantage to the less invasive approach. In the balloon studies, the North American trial included outcomes as early as 6 weeks, which demonstrated improvement in outcomes scores and range of motion compared with the control, vs. the study from England which reported the first time point at 3 months.
Everyday practice
If we are using scientific literature in our everyday clinical and surgical practice, we must be thorough in our evaluation of study design, patient population, methods and results, rather than just relying on the authors’ conclusions. When we publish, we are also responsible for acknowledging and discussing limitations of our work, requirements for future study, and areas where adequate conclusions cannot be drawn.
The value of the patient care can be improved if we are critical of the influence of published studies. We have to focus on studies with the highest quality of design and carefully advance our current use of innovative devices and incorporate new techniques that provide patients with the best possible outcomes.
- References:
- Metcalfe A, et al. Lancet. 2022;doi:10.1016/S0140-6736(22)00652-3.
- Verma N, et al. J Bone Joint Surg Am. 2022;doi:10.2106/JBJS.21.00667.
- Waterman BR, et al. Arthroscopy. 2021;doi:10.1016/j.arthro.2021.04.041.
- For more information:
- Anthony A. Romeo, MD, is the Chief Medical Editor of Orthopedics Today. He can be reached at Orthopedics Today, 6900 Grove Road, Thorofare, NJ 08086; email: orthopedics@healio.com.
- Nikhil N. Verma, MD, an Orthopedics Today Editorial Board Member, is professor and director, Division of Sports Medicine, and director, Sports Medicine Fellowship for Rush University Medical Center, Midwest Orthopaedics at Rush, and head team physician for the Chicago White Sox.
Collapse
Click Here to Manage Email Alerts