January 15, 2020
New Paclitaxel Meta-Analysis Questions Safety in CLI Population; Concerns Voiced as to Methods
January 15, 2020—In the wake of a year spent scrutinizing the finding of a mortality signal related to paclitaxel-coated balloons and stents in the superficial femoral artery (SFA), a new publication again calls the safety of paclitaxel into question, only this time in the critical limb ischemia (CLI) population. The results of a systematic review and meta-analysis of paclitaxel-coated balloons published today in the Journal of Vascular and Interventional Radiology (JVIR) by Katsanos et al echo those of the authorship group's December 2018 article in the Journal of the American Heart Association (JAHA), showing reduced need for reintervention but also lower amputation-free survival through 12 months.
It also appears likely that considerable debate surrounding the methods and conclusions will follow the publication, as it did the preceding study.
SUMMARY OF FINDINGS
In the current JVIR CLI publication, the authors present the findings of a systematic review and meta-analysis of eight randomized controlled trials (RCTs) studying the effects of paclitaxel-based drug-coated balloons (DCBs) compared with conventional balloon angioplasty (PTA) below the knee (BTK). The primary endpoint of the meta-analysis was amputation-free survival defined as freedom from all-cause death and major (above the ankle) amputation, marking a departure from the previous intermittent claudication/SFA meta-analysis, which focused solely on mortality. The secondary endpoint was target lesion revascularization (TLR).
Eight trials comprising a total of 1,420 patients, 97% of whom were deemed to have CLI, were included and analyzed through up to 1 year of follow-up. The meta-analysis included both published (five studies) and presented but unpublished (three studies) data, data with follow-up through either 6 (three studies) or 12 months (five studies), and both multicenter (five studies) and single-center (three studies) trials. The authors noted that tissue loss and foot ulcers were reported in approximately three-quarters of the population.
Amputation-Free Survival Findings
Regarding the amputation-free survival endpoint, the meta-analysis identified 74 deaths and 40 major amputations among 835 patients treated with DCBs, for a 13.7% crude risk of death or limb loss. In the 585 patients randomized to control arms, there were 39 deaths and 17 major amputations, for a 9.4% crude risk of death or limb loss. The authors noted that there was consistent evidence that amputation-free survival was significantly lower with DCBs, with a pooled hazard ratio (HR) of 1.25 (95% confidence interval [CI], 1.12–2.07). However, this was found to be driven by nonsignificant individual increases of both all-cause death (odds ratio [OR], 1.39; 95% CI, 0.94–2.07; P = .10) and major amputations (OR, 1.63; 95% CI, 0.92–2.90; P = .09). The number needed to harm was estimated to be 22 patients (95% CI, 11–94).
Although the authors determined that there was no significant heterogeneity between studies, nor evidence of publication bias, the quality of evidence was downgraded from high to moderate due either to imprecision driven by sparse data or the risk of incomplete/unpublished data.
TLR Endpoint Findings
In contrast to the safety elements explored in the primary endpoint, the efficacy of the DCBs studied was confirmed in the secondary endpoint, with a reduced need for TLR in DCB-treated patients. These results included 103 TLR events reported in the 875 cases treated with DCBs (11.8% crude risk) versus 159 instances of TLR in 620 control subjects (25.6% crude risk). The number needed to treat was eight patients (95% CI, 0.35–0.81; P = .004), although due to the significant heterogeneity of the included trials, the quality of evidence was again downgraded to moderate.
CONTEXT, COMMENTARIES, AND POTENTIAL RAMIFICATIONS
2019 was a year dominated by the discussion and fallout of the JAHA meta-analysis authored by Katsanos et al in December 2018, which found an increased late mortality risk in a largely claudicant population of patients treated with paclitaxel-coated balloons and stents in the femoropopliteal anatomy. Although no significant difference was seen between the study and control arms in the 28 trials evaluated at 1 year, a signal appeared in the 12 trials reporting 2-year follow-up and was observed to be even greater in the three trials that reached a combined 4- and 5-year endpoint. No causal link was determined by Katsanos et al in the paper, but the authors suggested the possibility of late toxicity related to paclitaxel. The safety concern sounded alarms among clinicians, professional societies, regulators, trialists, industry, and patients.
The FDA requested that all paclitaxel-based peripheral artery disease (PAD) device makers ascertain vital status for all enrolled patients to close a sizable gap in numbers of patients lost to long-term follow-up in the trials and convened a panel to explore the safety signal. Numerous studies were conducted to explore the safety of paclitaxel devices individually and in aggregate, attempting to determine whether the finding was statistical or causal in nature. Summaries of all associated studies and activities from the December 2018 JAHA meta-analysis through the present day can be found at evtoday.com/resource/paclitaxel-controversy. The devices remain on the market at present, although outcomes continue to be closely monitored.
Critics of the 2018 JAHA meta-analysis focused on several limitations and perceived flaws in its methodology and conclusions. Notable among these were its lack of patient-level data as a study-level meta-analysis, the considerable loss to follow-up in the long-term data of the trials comprising it, the mingling of unlike devices (DCB and drug-eluting stents) and their unique paclitaxel coating/excipient formulations, flaws in dose calculations, the likelihood that some control patients were treated with paclitaxel either contralaterally or for subsequent TLR, the potential presence of several forms of bias, and the insufficient powering of its included trials to drive long-term endpoints they were not designed to primarily assess, such as mortality beyond 1 year. Additionally, the deaths observed in the trials were wide-ranging in their recorded causes, and no deaths or adverse events were determined to be related to paclitaxel by the respective clinical events committees, although this determination is challenging in a patient population with comorbid conditions, increasingly so as data reach longer terms.
In their 2020 JVIR publication, the authors sought to explore the safety of paclitaxel-based devices in the CLI patient population, which comprised only a small portion of patients in the previous meta-analysis. All of the trials in the 2020 JVIR paper evaluated BTK device use, whereas the previous study solely focused on femoropopliteal lesions. Other notable distinctions include that only DCBs were included, and of particular note, the 2020 JVIR study included only trials with follow-up out to 12 months, which is notable in that the late mortality signal of the 2018 JAHA meta-analysis was not present at this interval.
The current CLI meta-analysis in JVIR was provided to Endovascular Today under embargo prior to publication. Under the embargo, Endovascular Today was able to solicit commentary from the meta-analysis authors, as well as other expert voices for inclusion in this coverage. The following represents key areas of discussion and critique provided by each respondent via separate interviews, as well as rationale and explanations provided by several of the meta-analysis authors.
Study- Versus Patient-Level Analysis
Addressing the authors’ decision to again employ a study-level model given the criticisms of this approach in response to the 2018 JAHA article, lead author Konstantinos Katsanos, MSc, MD, PhD, EBIR, of Patras, Greece, explained, “Our 2018 paper was a study-level meta-analysis of 28 randomized controlled trials documenting an increased long-term risk of death in case of the paclitaxel-treated subjects suffering from intermittent claudication. This was confirmed by the FDA investigations and the VIVA individual patient data meta-analysis of the subset of major industry-sponsored trials. Unless there is significant underlying intertrial heterogeneity (eg, clinical diversity—heterogeneous patient and/or lesion populations), the results of study-level and patient-level analyses are mostly in agreement. We therefore employed a study-level model again. I will also note that we have been refused access to patient-level data time and time again. Of course, individual patient data meta-analyses are advantageous in the interrogation of subset effects by avoiding Simpson’s paradox.”
Maintaining some of the more contentious study design elements of the previous meta-analysis while changing others was perplexing to several commentators speaking to Endovascular Today.
“Although the statistical methods used by the authors can be considered sound, the FDA in their further analysis has required the medical industry to provide patient-level data,” said Jos C. van den Berg, MD, PhD, of Lugano, Switzerland. “Still, in this new article, only study-level data are used. I believe that this will again lead to a lot of discussion in the upcoming months. Also, other aspects like the number of patients that were available for follow-up and the intention-to-treat form of analysis may again lead to renewed discussion.”
William A. Gray, MD, of Wynnewood, Pennsylvania, commented that “the total number of subjects in the analysis is far too small to make definitive statements without concern of a type I error.” As was the case with the 2018 JAHA meta-analysis methodology, Dr. Gray disagrees with the authors’ choice to use the total number of subjects enrolled as the denominators, rather than the numbers available for follow-up at 1 year, despite several of the published papers having CONSORT diagrams defining their patient flow. “The missingness of data is not always balanced between arms, as appears to be evident in the continuing data collection in the FDA SFA meta-analysis, which is narrowing the differences between groups,” he said.
Dr. Katsanos addressed this decision, commenting, “This is the standard approach in performing meta-analyses on the intention-to-treat principle. However, selection bias may be introduced if there are major differences in the rates of patients lost to follow-up or unavailable for follow-up between the active and control arms.”
Deciding the Endpoints: Mortality Versus Amputation-Free Survival
One of the concerns raised by several commentators discussing the 2020 JVIR publication with Endovascular Today was the change from an all-cause mortality endpoint in the previous meta-analysis to a composite amputation-free survival endpoint in the CLI study. Also at issue was the shorter duration of follow-up compared with the previous meta-analysis.
“When investigating novel endovascular devices, there are major differences in the hard clinical endpoints between studies for CLI patients and those for claudicants, as these should reflect the clinical symptoms that we are treating,” explained meta-analysis coauthor Stavros Spiliopoulos, MD, PhD, EBIR, FCIRSE, of Athens, Greece, when asked about the differences in designing trials for these populations. “Therefore, in the CLI population, the proper primary clinical endpoints are limb salvage and amputation-free survival, whereas in patients suffering from intermittent claudication, walking distance and reintervention-free interval are more appropriate endpoints.
“Another main difference is the time frame for which all the above mentioned endpoints should be investigated,” continued Dr. Spiliopoulos. “CLI patients are a severely morbid population with disappointedly high short- and midterm mortality rates, mainly due to cardiovascular death, while the majority of major amputations occur within the first year after diagnosis/revascularization. In contrast, claudicants demonstrate significantly better long-term survival and outcomes such as repeat procedures, and patency should be examined in the long-term in order to demonstrate a meaningful improvement in the patients’ quality of life.”
“My first reaction was that it was good to see a group looking to define the risks of paclitaxel devices in the CLI population,” said Ramon Varcoe, MBBS, MS, FRACS, PhD, of Sydney, Australia, agreeing that the CLI population is distinct from the claudicants evaluated in the previous meta-analysis and warrants specific investigation. “[However,] it must be said from the outset that with limited numbers and such short-term follow-up, care must be taken not to overreach in making conclusions. Moreover, taking a deeper dive into the meta-analysis revealed several methodological questions and important validity concerns.” In particular, Prof. Varcoe noted, “The authors showed no difference in all-cause mortality or major amputation rates [separately], but when they combined the two as the endpoint ‘amputation-free survival,’ they appeared to find a difference between DCB and PTA, with better amputation-free survival in the control group.”
Regarding the combined endpoint, Prof. van den Berg added concern as to the potential influence of wound care on outcomes. “We know from the IN.PACT DEEP study that wound care in multicenter studies is very difficult to standardize, while the DCB or balloon angioplasty treatment is easy to standardize in a protocol. Another issue that is not clear from this paper is whether the distribution of CLI severity was the same amongst the various studies,” he said.
When asked specifically for the reasoning behind the change in endpoint from 2018’s mortality to 2020’s amputation-free survival, Dr. Spiliopoulos reiterated the differences and respective clinical needs of the populations being studied. “Amputation-free survival is the relevant safety and efficacy endpoint in a CLI population,” he affirmed. “The main clinical issue here is how many patients are alive and not amputated. Notably, the significant difference in amputation-free survival noted in the paclitaxel-coated balloon group was driven by an increased rate of both major amputation (OR, 1.63; P = .09) and all-cause death (OR, 1.39; P = .10).”
Although seemingly in agreement regarding the disparate presentations and needs of the two populations, Dr. Gray came to a different conclusion. “From a larger perspective, the CLI population is generally an in-need population—one in which the mortality rates are high and need for reintervention is frequent,” he said. “Any amputation-free survival issues related to the use of devices would need to be based on rock solid data, which this is far from.”
Duration of Follow-Up
As previously mentioned, three of the eight included trials had follow-up of only 6 months, whereas the other five trials reported 12-month data. The safety finding occurring at this interval is in contrast to the SFA meta-analysis, wherein the mortality signal was observed at later follow-up but was not present at 1 year. On this point, Dr. Gray felt strongly that the disparity between the timing of the findings of the two studies affects the ability to posit a common biologic mechanism.
Offering a potential reason as to why the safety signal was seen earlier in the CLI study, coauthor Panagiotis M. Kitrou, MD, MSc, EBIR, of Patras, Greece, also emphasized the differences in the populations, specifically that the primary population of the 2018 study had fewer comorbidities and a lower cardiovascular risk compared with the population investigated in the BTK studies. “The majority of [the 2020 study] patients suffered from CLI and were at higher risk for limb or life loss,” he said.
Inclusion of Published Versus Presented Data
Similar to the follow-up distinction, three of the trials included have not been published, with their citations reflecting presentation at the CIRSE and LINC congresses. Commentators voiced concerns about the study’s lack of a requirement for peer-reviewed publication, with several wondering why longer-term published and unpublished data were not included, especially the IN.PACT DEEP 5-year data, which were recently presented. This major trial was instead represented by its 1-year published data, which were less favorable with regard to safety, ultimately resulting in the withdrawal of the studied device from the European market.
When asked for the authors' reasoning as to the inclusion of unpublished data, Dr. Katsanos elaborated on the paper’s assertion that this was done to reduce the risk of publication bias.
“This is a very important point and most pertinent to the examination of potential harm signals,” said Dr. Katsanos. “Publication bias occurs when investigators refrain from publishing clinical trials that either do not show efficacy or document harm. Hence, it was of particular importance to include all randomized studies regardless of publication status to provide the most comprehensive outlook of the scientific question at hand. As explained in our recent 2020 paper in detail, we indeed found evidence of small study effects when analyzing deaths and amputations individually, which may imply the presence of publication bias. We therefore expect more data to come to light over time. Again, sensitivity tests showed consistency when examining published and unpublished data.”
“In the previous meta-analysis, three studies were cited at 4 to 5 years, one of which was only presented (IN.PACT SFA), and this was already something that disturbed me,” commented Prof. van den Berg. “In the current meta-analysis, three out of eight studies have not been published, and data-analysis cannot be considered optimal when taking the numbers from single slides presented during a conference. I think such studies should not be allowed in a meta-analysis, mainly because these papers have not undergone peer-review and evaluation for statistical correctness.”
“We must take great care in including data that has not been peer reviewed,” said Prof. Varcoe, noting that it is common for event rates to change between presentation and final publication, which may alter the findings of a meta-analysis. He believes this is particularly true for low-frequency endpoints such as amputation-free survival at 6 to 12 months, and that it is also difficult for reviewers to determine the quality of evidence from a short congress presentation as opposed to a detailed manuscript.
“Including unpublished data is not unheard of, particularly when there are few published studies in the literature,” continued Prof. Varcoe. “What concerns me most is the glaring omission of the most significant study of DCB use in BTK arteries of CLI patients. The 5-year results of the IN.PACT DEEP study were presented by Prof. Thomas Zeller at the AMP Symposium in August . Like the other three studies, it was unpublished in September 2019 when the systematic review was conducted. However, if you include some unpublished studies, you must include all.
“IN.PACT DEEP was a rigorously conducted trial, which fulfilled all of the inclusion criteria for the meta-analysis, as evidenced by inclusion of its earlier 12-month results,” continued Prof. Varcoe. “In that study, amputation-free survival was numerically better in the DCB group (DCB, 135/239 [56.5%] vs PTA, 65/119 [54.6%]). The inclusion of such a study is important for two reasons: It provides long-term follow-up, which was linked to mortality in the original meta-analysis in claudicants, and it had a high number of events (death and amputation). Statistically, meta-analysis gives additional weight to large studies with high event rates because observed differences are less likely to be due to chance. It is my view that if those contemporary results had been included in this meta-analysis, the conclusions regarding amputation-free survival and safety would have been very different. Given the reintervention-reducing benefit of DCB the authors have demonstrated, we must get this right. A serious omission such as this must be corrected if we are to determine the truth. Our very patients’ lives depend on it.”
Addressing questions as to the shorter duration of follow-up and the omission of unpublished data that were available within the period the meta-analysis was conducted, the authors stood by the use of the shorter endpoint but also noted they were not previously aware of the IN.PACT DEEP 5-year data presentation.
“The authors would like to point out that all other studies had only short-term follow-up available ranging from 6 months to 1 year,” said Dr. Spiliopoulos. “Moreover, the authors were not aware of the unpublished 5-year IN.PACT DEEP data at the time of their literature search. In addition, as previously discussed, 1 year is a meaningful time frame for revascularization outcomes in the CLI population.”
Evaluating DCBs Alone
Regarding why the current meta-analysis included only DCBs, whereas the 2018 study also included DES, Dr. Kitrou explained, “There is only one study available with a paclitaxel drug-eluting stent (the PADI randomized trial) including very short lesions, approximately 2 cm, compared to the longer lesions treated with paclitaxel DCBs, which were in the range of 9 to 18 cm, approximately. Thus, the patient population and the lesions were not homogenous, and this would have introduced potential heterogeneity, contaminating the results.”
As to the study’s inclusion of paclitaxel but not other antiproliferative-coated devices, Dr. Kitrou commented, “Our interest was on paclitaxel, as we were sensitized from our previous meta-analysis, where the mortality harm signal was strong with the specific drug. Again, trying to include a homogenous patient sample, our second meta-analysis was focused only on DCBs in below-knee arteries for CLI treatment.”
Dose calculations were an area of debate after the SFA meta-analysis, with criticisms including that without patient-level data, the results were based on the study device's overall formulation rather than the quantity of drug each patient actually received based on the numbers and lengths of lesions treated and/or numbers and lengths of devices used.
Dr. Gray took particular issue with the dose-response analysis and discussion. “It simply assigns balloon dosing, does not take into account lesion length treated, includes only one study with a ‘low-dose’ device, and compares that to the other ‘high-dose’ devices,” he said. “Beyond these issues, the methodology is different than the one used in—and defended after—the original paper.”
“There was overall a lot of confusion and misunderstandings about our SFA dose response meta-analysis,” began Dr. Katsanos when asked by Endovascular Today about these concerns and how they were addressed in the current study. “We performed a rudimentary analysis, but we have demonstrated a significant dose-response relationship both at 2 and 5 years on follow-up presentations and in the 2019 FDA panel meeting as well. In case of our BTK meta-analysis, we only tested high-dose (3.0–3.5 μg/mm2) devices and low-dose (2.0 μg/mm2) study strata, because the small number of RCTs included and the absence of individual patient data precluded a more meaningful quantitative meta-regression analysis.”
Prof. Varcoe also raised the question as to whether and when the authors filed their protocol on the PROSPERO database, which helps to assure readers that meta-analysis endpoints are determined a priori and not after the analysis. At the time of review, he was not able to find the study's methods on the database.
When asked by Endovascular Today whether this step had been taken, Dr. Katsanos responded that the authors submitted a registration application with PROSPERO in September 2019, but that it is still being processed by their editorial team.
DATA IN CONTEXT
2019 saw the halt of several studies involving paclitaxel devices in PAD and the start of many more, with the former mostly ongoing randomized trials and the latter predominantly prospective analyses of previous trials, registries, and large-scale databases. A number of these studies have already been published and presented over the course of a conference schedule dominated by exploring and debating the paclitaxel controversy, although their findings were not presented in detail or contrasted in the JVIR meta-analysis discussion.
Putting the current meta-analysis into the context of recently published studies, which use different populations, designs, and methods, Prof. van den Berg highlighted a few examples, including one just published in January.
“This meta-analysis is based on data from [randomized controlled trials] and comes to an entirely different conclusion as compared to the data that were published by Secemsky et al in a large cohort of patients with CLI from a Medicare database where no difference was seen in mortality,” he said. “Another recently published study based on German insurance data by Behrendt et al even showed a benefit of paclitaxel-eluting devices not only with respect to mortality but also amputation-free survival.”
In contrast to the 2018 JAHA SFA meta-analysis, the 2020 JVIR CLI publication addresses a device class with limited market penetration primarily involving use in clinical trials in recent years. As previously mentioned, the In.Pact Amphirion drug-coated balloon (Medtronic) was recalled from the European market in December 2013 based on 1-year results that missed the primary efficacy endpoint and included a trend toward amputation. In August 2019, BD reported that the FDA had deemed its premarket approval application for a BTK application for its Lutonix paclitaxel DCB to be “not approvable in its current form,” although this application may still be pursued in the future. DCBs for use in femoropopliteal and arteriovenous access remain on the market.
As a result, the authors speculate that the impact of the current paper will not be as profound as in the SFA landscape, said Dr. Spiliopoulos.
Is Consensus on Paclitaxel Achievable?
One of the challenges in interpreting a meta-analysis is in evaluating the potential effects of the heterogeneity of its source data. Disagreements as to which data sources should be included and excluded, and why, persisted from the first 2019 podium presentations in Leipzig and South Florida to the last lectures in Sydney this past December. To better understand the disparity of opinions between the authors and the critics of their methods, we asked Dr. Katsanos to explain how his group determines the threshold at which heterogeneities such as those described here are beneficial or detrimental to producing relevant data.
“It is often argued that every meta-analysis is only as good as the data that it pools,” answered Dr. Katsanos. “This is why a meta-analysis of high-quality randomized controlled trials lies at the top of the pyramid of modern evidence-based medicine. To that end, we only searched for RCTs, and we also used the PICO (Patient, Intervention, Comparison, Outcome) tool to frame the scientific question at hand. Use of the PICO tool helps limit underlying heterogeneity and bias in meta-analysis. In our case, we defined the scientific question as follows: ‘In patients suffering from critical limb ischemia, is treatment of the infrapopliteal arteries with paclitaxel-coated balloons, compared to conventional balloon angioplasty, safe and effective in improving clinical outcomes?’”
It remains to be seen whether the vascular community will agree that the current meta-analysis sufficiently addresses the question. Acknowledging that the criticisms shared in this coverage represent the opinions of a small sample commentators who were provided with the data in a brief window prior to publication, controversy may be again be more likely than consensus.
“This paper moves us backward in our understanding of the risk-benefit ratio of paclitaxel-coated balloons for peripheral artery disease,” said Peter Schneider, MD, of San Francisco, California, providing a summary statement upon review of the 2020 JVIR CLI meta-analysis and voicing concerns for the potential impact of its publication in light of the continuing fallout from the 2018 JAHA meta-analysis. “Published approximately 1 year after the last paclitaxel meta-analysis from this group, it does not appear to take into account what we have learned. We know from the events of the past year that data quality is essential, even for summary-level meta-analyses. When data is incomplete or extrapolated, an overwrought reaction can be generated. This has been well demonstrated by the curtailed use of DCB and DES in the face of the incredible shrinking hazard ratio for paclitaxel devices in the femoropopliteal segment and the inability to demonstrate any danger signal in massive amounts of real-world data.
“We also know that the paclitaxel issue is highly sensitive and that additional data should be held to a high threshold. Multiple communities are relying on us to be judicious about what we publish (patients, hospital administrators, etc), and we need to be able to guide practice decisions. Publishing something that relies upon substantial amounts of unpublished data, that fails to include the largest trial with the longest follow-up, and has multiple flaws, is ill-advised in the current environment and should be better refined before committing to the scientific literature,” said Dr. Schneider.
“These results are under a cloud due to methodological questions and the omission of the most important study,” summarized Prof. Varcoe. “This should not impact anyone’s practice until those issues have been addressed.”
“I think that there is a clearer picture now about potential toxicity related to use of paclitaxel-coated devices,” concluded Dr. Katsanos, in contrast. “Nontarget paclitaxel embolization is a plausible mechanism and the most reasonable culprit for its potential adverse local and systemic effects. This is even more important when treating CLI patients, who are clearly at higher risk of limb or life loss.”
Despite early and persistent criticism of the 2018 JAHA meta-analysis and little in the way of consensus regarding its findings and conclusions, the publication was among the most impactful of a generation. The explorations of its safety signal also saw unprecedented collaboration, and several vital needs for improvement in the design and follow-up of peripheral clinical trials were agreed upon by the authors, regulators, investigators, and industry alike.
It is unclear if the same will be true as another calendar year begins with the same debate as the last, but with the 2020 conference season about to begin, it appears the paclitaxel controversy will continue to have a prominent role at the podium and on PubMed.