Raising Critical Questions
Sumaira Macdonald, MBChB (Comm.), FRCP, FRCR, PhD, discusses the questions she posed to the EVA-3S and SPACE presenters at VEITH, the difference between failure to show noninferiority and superiority, and critical aspects of future trial design.
Endovascular Today: After the presentation of the EVA-3S and SPACE trial data at the VEITH Symposium in November 2006, you posed questions regarding some of the findings and conclusions. What was the reason for your concern?
Dr. Macdonald: I posed two questions regarding the 30-day results of EVA-3S. The first was a query regarding the trial's requirement for the periprocedural dual antiplatelet regimen. In SAPPHIRE and SPACE, combined aspirin and clopidogrel with appropriate preloading and 28-day post-stenting course were mandatory, just as they are in the ongoing trials, CREST and ICSS. EVA-3S simply recommended this protocol, and yet there is level-1 evidence in support of the importance of this regimen, not only in the setting of coronary intervention (CREDO and PCI CURE trials),1,2 but also for CAS.3 The stroke rate is significantly higher when aspirin and heparin are employed in place of a dual antiplatelet regimen.
It is a little strange that the EVA-3S safety committee should overlook this fact and instead choose to mandate the use of cerebral protection when we have, at best, level-3 evidence for routine use of these devices from the available literature. Furthermore, the data on which the committee made its decision are decidedly weak; the absolute numbers of procedural strokes were three in the protected patients (3/58; 5.2%) and two in the unprotected patients (2/15; 13.3%), reaching significance only because of the differences in size of the denominator. Furthermore, a substantial number of events were nonprocedural and could not, therefore, logically be prevented by use of a protection device. There was, of course, no randomized comparison of protected and unprotected stenting.
My second question was aimed at evaluating the level of experience for those performing CAS within EVA-3S. The investigators consistently point out that there were no significant differences in outcome between those centers performing fewer than 21 procedures, 21 to 40, and more than 40 procedures. To try to state that experience makes no difference in outcome for CAS flies in the face of both logic and the literature. CAVATAS clearly showed the influence of experience on outcome for both CEA and CAS (only 26% of patients received stents in this landmark trial). Furthermore, the fact that a center is high-volume does not mean that every interventionist (surgeon, radiologist, or cardiologist) is experienced. It would be more meaningful to compare results of individuals rather than that of centers. CAS is a complex intervention and, like CEA, is unsafe when performed in low volume.
Let us compare "like" with "like." EVA-3S mandated that surgeons performing CEA should have performed 25 of these procedures in the preceding year. We can assume that those operating within the trial had, on the whole, been qualified to do so for at least 1 year and therefore had probably performed more than 25 CEAs. Those performing CAS had to have performed a total of 12 CAS procedures or five CAS procedures plus 35 stenting procedures in the supra-aortic trunks, which is technically an entirely different procedure with much lower risk. Any experienced carotid stenter will point out that the learning curve for CAS is probably well in excess of 40 cases and may even be more than a couple hundred,4 especially for surgeons performing CAS, as they are traditionally less comfortable with catheter/guidewire techniques.
I asked why 12 patients were "crossed over intraoperatively to CEA" from CAS within the trial. This seems a little high. The possible reasons for this are controversial. As a vascular/interventional radiologist, I could not "cross a patient over intraoperatively," as I do not know how to perform CEA and neither does an interventional cardiologist. The inference is that, first, in 12 cases, the decision to stent was inappropriate (therefore, that the operators were inexperienced in patient selection for the technique); a learning curve involves more than technical ability. Second, that the operators involved in these "cross-overs" were vascular surgeons. The trial had several operators listed as VSI, or vascular surgeon interventionist.
Endovascular Today: Did you feel your questions were adequately addressed by the presenters?
Dr. Macdonald: Unfortunately, my second question was not answered as the speaker stated that he did not understand it. In response to my first question, it was pointed out that there was no significant difference in outcomes for those patients on dual and single antiplatelet regimen. Quite clearly, this is a nonrandomized comparison of patients treated with and without "standard of care" antiplatelets and represents a tiny subset analysis.
Endovascular Today: How would you describe the response to either or both of these studies in the UK to date?
Dr. Macdonald: Physicians and health care providers in the UK value and are heavily influenced by evidence-based medicine. Within the National Health Service, we are fortunate because reimbursement and litigation issues are less relevant than in other health care settings.
Accepting this, CEA is an operation that is very much enjoyed by vascular surgeons, and in the UK the procedure is performed by vascular surgeons and occasionally neurosurgeons. CAS is performed largely by interventional radiologists, some interventional neuroradiologists, two vascular surgeons, to my knowledge, and a small but growing number of interventional cardiologists. There is, of course, the potential for personal agendas to influence opinion and practice. I have heard a senior vascular surgeon describe SPACE as the death knell for CAS. If we wish to be controversial about it, we could describe the situation as a boxing match between an enthusiast in one corner and a Luddite in the other. Reason rests somewhere between the two extremes. Most stroke physicians and neurologists with an interest in stroke prevention are candid about the results of SPACE and EVA-3S, both of which were prematurely stopped, thereby leading to a potential overestimate of procedural risk for CAS, that is, data analysis occurring at a random high and are willing to continue to randomize within ICSS. Certainly, this is the case for those few centers with relatively high CAS throughput. For centers intending to start a CAS regimen, there is on the whole more apprehension than was previously the case before publication of the 30-day results of EVA-3S and SPACE.
Endovascular Today: Have the data slowed the trend toward stenting of carotid artery stenosis in favor of CEA? In your opinion, why or why not?
Dr. Macdonald: With the caveat of novice centers starting a carotid stenting program, the data have not slowed carotid stenting in the UK, and those with a good grasp of randomized trials see the value of supporting trials to completion. Those who are considered to be at high risk for CEA—and there is no consensus on "high risk"—will continue to be offered CAS if there are experienced interventionists available. Outside this indication, patients will be randomized within ICSS. With the exception of a tiny minority of centers, very few symptomatic low-operative-risk patients receive stents outside of trials, and very few asymptomatic patients are stented in our relatively conservative environment.
Endovascular Today: What conclusions do you feel can be drawn from each of these trials?
Dr. Macdonald: Regarding EVA-3S, would it not be ethically very difficult to offer CAS to a low-operative-risk symptomatic patient in France? However, I think EVA-3S proves that CAS should be performed by physicians who are not just technically experienced but who are also adept at the evaluation of anatomic suitability for CAS. In EVA-3S, interventionists used five different stents and seven different cerebral protection devices, and experience with only two procedures was required for any new device used. By comparison, in CREST, 1,472 patients were enrolled in a lead-in phase that required training programs of up to 20 procedures per investigator using a single stent and protection system. Quite simply, CAS should be performed by properly trained and experienced people—this is clearly not rocket science.
Regarding SPACE, I would first like to quote Professor Nick Cheshire, who stated at Controversies and Updates in Vascular Surgery in January 2007 that, "failing to show noninferiority of CAS is not the same as proving superiority of CEA." I think we should be mindful of this fact.
We must also bear in mind that this trial was prematurely stopped for reasons of futility and lack of funding. This trial, designed as a noninferiority trial, mandated a preset value of delta (the maximum acceptable difference in outcomes between the two treatments offered). The upper limit for the margin of noninferiority was preset at +2.5. The trial was originally powered for 1,900 patients, but at around 1,100, patients it became clear to the statisticians and data-monitoring committee that the trial would have only a 50% chance of showing noninferiority for CAS at the original target of 1,900 patients, but would have an 80% chance of demonstrating noninferiority at 2,500 patients. Without funding to support ongoing recruitment, the trial was stopped. In SPACE, although there was no significant difference between CAS or CEA for either analysis, there was a slight trend toward CEA. Clearly, a trend is a very soft outcome measure in the context of a randomized trial and should be interpreted with caution. Because the upper confidence interval limit for the primary outcome was more than 2.5, the study failed to show noninferiority for CAS. This does not mean that superiority for CEA was shown.
For superiority to be shown, the 95% confidence interval for the treatment effect should lie not only entirely above delta (+2.5 in this instance) but also above zero. In switching from noninferiority to superiority constructs, the intention to treat analysis assumes greater importance, within a noninferiority trial, both intention-to-treat and per-protocol analyses should be given equal weighting. From any perspective, in SPACE, both intention-to-treat and per-protocol confidence intervals crossed zero (ie, the superiority of CEA was not shown).
To quote Professor Ross Naylor, of Leicester Royal Infirmary, "In other words, surgeons will conclude that carotid angioplasty and stenting was inferior to carotid endarterectomy, although interventionists will conclude that there was no significant difference."5
Last, Andreas Kastrup, of Gšttingen, Germany, in addressing the audience at the International Symposium on Complications After Endovascular Repair of Aneurysms and Carotid Artery Stenosis in December 2006, indicated that in SPACE, the stroke rate was lower in the CAS limb than in the CEA limb before routine cerebral protection was employed. He added, "This will never be published." There is a learning curve also for the safe use of cerebral protection systems, and it remains possible that interventionists who were accustomed to unprotected CAS exposed SPACE trial patients to their learning curve for cerebral protection.
Endovascular Today: What are your feelings concerning the SAPPHIRE trial? What would you consider to be its strengths, and in which areas did it fall short?
Dr. Macdonald: Concerning SAPPHIRE, the primary outcome was combined stroke, death, and myocardial infarction (MI). Outcomes for symptomatic high-operative risk patients were significantly better for CAS than CEA, resulting largely from the differences in MI between treatment limbs. There were no significant differences between stroke and death. Some have criticized the "creative" use of this composite endpoint on the basis that traditionally, trials of CEA against best medical therapy have not included MI, but pragmatically, MI is important to the patient. Although the stroke/death outcomes for CAS for symptomatic patients were within the American Heart Association (AHA) Guidelines (≤6% all stroke/death), the outcomes for CAS in the asymptomatic population (the majority) were higher than the cut-off given by the AHA for asymptomatic patients (ie, stroke/death should be ≤3%). In SAPPHIRE, the 30-day stroke/death rate for asymptomatic patients, although lower in the CAS limb than in the CEA limb, was 5.8%. We must remember that the risk of stroke in medically treated asymptomatic patients is around 2% per annum, and arguably, we are doing more harm than good when intervening in high-risk asymptomatic patients either by CAS or CEA.
On another quite separate note, regarding patients thought to be at high surgical risk due to significant coronary artery disease, it remains entirely possible, but unsubstantiated, that percutaneous coronary intervention could have been performed by interventional cardiologists involved in SAPPHIRE, at the same sitting, immediately preceding CAS. While this might seem unfair in the setting of a trial that includes MI in a composite endpoint, in reality, it is perhaps an advantage of endovascular management of carotid artery disease—that the coronaries can be stented "on the way up."
SAPPHIRE's main strength was that it was the first to show definitively that CAS is less cardiopathic than CEA. Although this had been considered to be the case anecdotally, it had not previously been demonstrated in a purely clinical setting. Trial weaknesses include the fact that it was also prematurely stopped (largely due to increasing resistance from referring clinicians and patients to randomize), the involvement of industry in the design of this commercially sponsored trial, and the fact that 400 patients were excluded from the trial because they were considered too high-risk, surgically.
Endovascular Today: How can future trials address lessons learned in previous studies?
Dr. Macdonald: I think we should support trials that are funded by independent national funding bodies. For example, the recently launched Asymptomatic Carotid Surgery Trial 2 (ACST-2), comparing CAS and CEA for asymptomatic patients with significant carotid disease is funded by Health Technologies Assessment, CREST is funded by the National Institutes of Health and National Institute of Neurological Disorders and Stroke, and ICSS is funded by the Medical Research Council. We should strive to support such trials to completion.
Second, we owe it to the patients recruited in future trials to include only experienced operators/interventionists, even if this means relatively slow recruitment and stringent proctoring and credentialing programs. Recall that 40% of interested centers were rejected by the technical management committee of ACAS on the basis of limited experience, and that those centers with poor outcomes within the trial were prevented from recruiting additional patients. When EVA-3S was launched in 2000, there were only four centers in France capable of offering CAS, and because recruitment was painfully slow, other less-experienced centers were subsequently allowed to recruit, as stated by Patrice Bergeron, MD, at the International Symposium on Complications After Endovascular Repair of Aneurysms and Carotid Artery Stenosis in Leuven, Belgium, December 2006.
Third, if the need arises to reinterpret the results of a noninferiority trial as a superiority trial or vice versa, then the caveats and limitations of so doing are made transparent to the readership of any journal in which these trials are published and that the methodology and limitations of any trial that could have profound implications for a technique such as CAS be fully divulged.
Endovascular Today: What expectations do you have from trials that are currently underway, such as ICSS and CREST?
Dr. Macdonald: ICSS, (or CAVATAS-2), an international trial comparing CAS and CEA for low-risk symptomatic patients has recruited more than 1,100 patients as of January 2007, and has a target of 1,500. The safety committee has seen no cause for concern, and we can expect this trial to run to completion. CREST, including asymptomatic patients, similarly has posed no problems for their safety committee. It is unlikely that the results of these trials will, in isolation, sway public opinion worldwide, but their results will add to a bank of level-1 evidence and, by combining the results of all randomized trials involving CAS, allow meaningful subset analyses that may point to particular strengths and weaknesses of CAS with respect to patient, lesion, and clinical variables.
Sumaira Macdonald, MBChB (Comm.), FRCP, FRCR, PhD, is a Consultant Vascular Radiologist and Honorary Clinical Senior Lecturer at Freeman Hospital, Newcastle upon Tyne, England. Dr. Macdonald may be reached at firstname.lastname@example.org.