Meta-analysis: Paclitaxel-coated balloons may be harmful in below-the-knee lesions

The design of this study confuses me. Amputation-free survival is an appropriate efficacy endpoint for CLI trials. It helps us to answer the question as to whether our treatment has been effective at keeping our patient alive longer and avoiding an amputation, two things that are very important in the CLI population. However, the question these authors should have been asking on the back of their previous study was “is paclitaxel toxic to the body or to the leg?”. This would mean defining the primary endpoint as either mortality or major amputation. Both of these were evaluated as secondary endpoints and found no association with PCB. Unfortunately, because the authors failed to register their protocol on PROSPERO before the stage of data extraction, as they have done in previous systematic reviews, we will never know whether they designed their study a priori or whether they have trawled data to find a statistically significant endpoint.

The most concerning aspect of this study are that almost half the randomized controlled trials which were included were from unpublished data. There are several problems with including data which has not been peer-reviewed. The first is that event numbers frequently change when the data are finalized and cleaned prior to the publication. That is an issue when a few data anomalies can change the results of a meta-analysis dramatically when you’re looking at a low frequency endpoint like amputation-free survival at 6 to 12 months. We also do not know whether those studies provide a high quality of evidence, and it is impossible to determine the risk of bias within the meta-analysis. A meta-analysis is of value only if the risk of bias is low or moderate. Finally, it has not had review of its statistical methods to determine whether they are rigorous and appropriate. The authors of this meta-analysis claim they included those studies to avoid publication bias, but all three unpublished studies are in the public sphere and are being reviewed with a view toward publication. Would it not have been appropriate to wait for those finalized data sets and determine the quality of evidence?

Unfortunately, while the authors included three unpublished studies, they completely overlooked the most important unpublished study. Keeping aside what I just said about unpublished studies, if you choose to include them, you must include all of them. The 5-year results of IN.PACT DEEP were presented in August at AMP: The CLI Meeting, which fell inside the period of the systematic review. The 12-month results were published previously, so we know those have been peer-reviewed and are of high quality. The authors included 12-month results, so we know it fulfilled their inclusion criteria. As it has 5-year follow up, if we are to believe the long-term toxicity explanation of their 2018 meta-analysis, it is just this study that should be included, yet it was omitted, with the authors claiming that they were “unaware of its existence.” This is one of the strangest things I’ve ever come across in the world of meta-analysis and it defies plausible explanation.

I took steps to repeat their meta-analysis, using exactly the same data for amputation-free survival, making just one change. I updated the IN.PACT results to include the numbers presented in August. This changed the findings dramatically. There was no significant difference in amputation-free survival between the PCB and uncoated balloon arms (RR = 1.08; 95% CI, 0.89-1.31), illustrating how a single omission can completely change the findings and alter the conclusion. Keep in mind that researchers have demonstrated reduced reinterventions with this technology, so we know it has the potential to benefit our patients. Claiming there’s a safety concern based on incomplete data has the potential to harm patients through restriction of the use of the drug-coated device.

I do not believe the findings are sound. However, if we are to believe that there is no safety signal in claudicants at 12 months, based on those same authors’ 2018 meta-analysis, then finding a safety signal at this short-term timepoint in patients with CLI would make no sense. Yes, it would be surprising and difficult to explain.

Unfortunately, the implications for paclitaxel-coated devices are dire. They were already under a safety cloud and this will add further weight to the voices of those who do not believe in this technology. This is why we have to take such great care to perform studies like this properly, leaving no stone unturned. We have a great responsibility as thought leaders and scientific researchers. It is not good enough to perform an important meta-analysis such as this and simply say that you were unaware of the most important study that exists.

We have a similar meta-analysis in press with the Journal of Endovascular Therapy. Ours also examined mortality after the use of paclitaxel-coated devices in patients treated for CLI. We kept the methodology very similar to the 2018 meta-analysis to answer the question about whether there is a mortality safety signal seen in this group of patients. I am hopeful that those results will add value to the ongoing debate.

This meta-analysis is not an ideal study of paclitaxel safety in below-the-knee lesions. The analyzed trials included small single-center series without any independent clinical events committee, as well as large multicenter independently adjudicated studies. The two large studies included devices that are not currently available for clinical use because they demonstrated no clinical benefit to the patients treated.

I find the conclusions to be weak at best. Specifically, the finding of increased mortality with paclitaxel-coated balloons for below-the-knee lesions was not statistically significant. The finding of increased rates of amputation with paclitaxel-coated balloons for below-the-knee lesions was not statistically significant. It was only when these two outcomes were combined that any statistical significance was realized.

The fact that a group of random trials evaluating technology that has not demonstrated clinical benefit to date could be interpreted to derive potential harm is not surprising. What is surprising is that a trial with long-term follow up of paclitaxel-eluting stents in the below-the-knee circulation (PADI) that demonstrated clinical benefit and no difference in mortality out to 5 years was not chosen to be included in this meta-analysis. 

In the United States, there are no paclitaxel-coated balloons available for clinical use in the below-the-knee circulation and, as a result, all use of these devices should be under the auspices of a prospective research trial evaluating the efficacy and safety of the technology.

We vascular specialists are all concerned about the safety of our patients and are especially troubled by the high rates of mortality and major amputation in patients diagnosed with CLI. We must all work together to study, using the best scientific methodology, any technology that can improve the quality of our patients' lives and help keep them alive with their limbs intact.

I am eagerly awaiting the results of the SAVAL BTK trial, as well as future studies evaluating new drug-coated balloons and scaffolds.