CAS Outcome Reports
A reader’s guide to critical assessment of the literature.
The past 15 years have been witness to the birth of an entirely new therapy for extracranial carotid artery disease. From its initial configuration of a tracheobronchial Wallstent (Boston Scientific Corporation, Natick, MA) and 0.035-inch-based equipment without embolic protection practiced by a small fraternity of operators from a variety of specialty backgrounds, carotid artery stenting (CAS) has matured significantly with dedicated carotid equipment from stents to distal filters to proximal flow reversal, which is now more widely disseminated. There have been no fewer than 10 US Food and Drug Administration (FDA) approval/clearance studies and several postmarket multicenter prospective studies representing well over 12,000 procedures reported in the literature in a short span of 7 years. In other words, a legitimate field, not only of novel therapy but also of clinical study, has emerged.
As a result, for both the producer and the consumer of literature regarding CAS, there may be significant challenges in understanding the critical issues related to this therapy. Because a device therapy is ultimately being compared to a long-standing surgical standard of care, there are many opportunities to draw incorrect conclusions (about either therapy) due to differences in reporting standards, trial design and conduct, populations being analyzed, etc. As a corollary example, results from drug-eluting stents are typically segregated by complexity of lesion and patient type (ie, diabetics, small vessels, bifurcations, long lesions, left main, etc.), and outcomes in the more complex groups are expected to be different and generally worse than those in the nondiabetic, single-vessel, de novo, > 3-mm intervention. Unfortunately, the same separation has not been strictly maintained for CAS outcome analysis, and because CAS had its roots in clinical practice mainly as an alternative to patients who were at higher-than-usual risk for carotid endarterectomy (CEA) or were excluded from previous landmark CEA trials, the population for whom data are available in CAS is largely composed of “confounders” or patients who would otherwise affect CEA outcomes negatively. In general, this is an older group of patients with a significant severity of illness for whom there are little or no quality CEA data available for even historical comparisons.
This article intends to examine some of the issues of CAS outcome reporting by using specific examples of studies—both positive and negative—as the foundation for critique and discussion.
TRIAL CONSTRUCT AND CONDUCT
In carotid revascularization, CAS or CEA, it is generally acknowledged that there are material differences in outcomes between symptomatic and asymptomatic patients, high- and standard-surgical-risk patients, as well as patients younger and older than 80 years of age. So when evaluating the results of any given study, and to fit it into an appropriate historical context for comparison, the study cohort must be defined by those factors.
Because event rates for CAS and CEA are low, comparisons between the two therapies generally require well in excess of 1,000 patients in order to possess adequate statistical power. Moreover, if one were attempting to assess the effect of a change in devices that would lead to a 1% improvement, a trial of over 5,000 patients would be necessary. There is currently only one1 published randomized trial > 1,000 patients (1,200); however, it was halted without a definitive statistical preference for CAS versus CEA (see later section).
Prospective Versus Retrospective Data Collection
It is axiomatic that prospective data collection and prespecified statistical analyses are the standard in trial design. However, it is not always readily obvious what effects retrospective reporting may have, especially in a comparative analysis (CAS vs CEA). A retrospective study inherently introduces the bias of the operator of one therapy over another for a certain patient or lesion (ie, selection bias), does not allow for uniform outcomes assessment (ie, some patient groups may have variability in the degree or amount of standard neurologic evaluation performed), and relies on site and operator reporting event types and severities rather than a clinical events committee following specific definitions and rules.
Although there are certain adjustments for severity of illness and other covariate differences (ie, propensity analysis) that can be performed to attempt to account for the selection biases, in the end, retrospective studies in CAS will generally suffer from the fatal flaws of disparate populations and inadequate event detection and reporting.
It is also worth noting that these trials were performed as multicenter studies. Single-center results, while often laudable, do not demonstrate the generalizability of the technique.
Centralized Qualifying Committees
In any prospective randomized study between the two therapies, it is important that the operators be sufficiently qualified and experienced so that they do not add to the results as confounders. This was the case with the landmark studies of CEA from the 1980s and 1990s that included only the best surgeons with demonstrated best results as study investigators. Such diligence in site selection vetting for CAS studies has not always been followed, with predictably catastrophic outcomes. This is likely because CAS was not an established therapy, and many trials started before an adequate judging of the interventionists was done, although this is not the case with the CEA operators. However, in other trials, no attempt at standardized evaluation or training of potential interventionists was even undertaken.
Independent Evaluation of Outcomes
It has been amply demonstrated that an independent assessment of the procedural outcome by a neurologist will identify minor neurologic events not typically noted by the operators. Any study in which this was either not done, or not uniformly done, will suffer from underreporting of events, and therefore, will be an improper assessment of the therapy being tested.
Site-reported events require adjudication using a clinical events committee and predefined rules. Without such a mechanism, there is generally an underreporting of events.
The standard use of stents and embolic protection devices (EPDs) is a minimum requirement for a CAS arm of a trial reporting results. Ideally, operators would be experienced in the device(s) selected for the trial. Dual-antiplatelet therapy, for at least 1 month, should also be stipulated.
In an effort to determine differences between treatment groups (CAS vs CEA) or devices (open cell vs closed cell), many studies will perform ad hoc statistical analyses. At best, the results of such analyses are hypothesis generating but cannot and should not be the basis for definitive statements or changes in practice.
Prospective Randomized Studies
There are three published multicenter randomized trials (SAPPHIRE,2 SPACE,1 and EVA-3S3) studying CAS and CEA (CAVATAS is primarily an angioplasty trial), with CEA beingused as the standard of care for comparison purposes. Although randomized trials generally represent the highest level of clinical science, each of these trials has critical flaws that limit their import.
SPACE. A standard surgical-risk trial of symptomatic patients randomizing CEA and CAS 1:1, had neurologic audit, central vetting and adjudication committees, and prespecified appropriate analyses. Although after 1,200 patients were enrolled, there was no difference in the primary endpoint of 30-day death and stroke. SPACE was originally projected to require 1,900 patients but was halted after sponsorship was withdrawn when slightly higher event rates than originally assumed in both arms increased to > 2,500 to prove noninferiority. Unfortunately, EPDS were not used in 73% of the patients, so conclusions regarding protected CAS cannot be made. Moreover, the authors’ summation was incorrect, “SPACE failed to provide noninferiority...” when in fact, the only correct conclusion can be, “SPACE failed to reach its original and subsequent enrollment goals, and therefore, no definitive statements can be made.”
EVA-3S. A standard surgical-risk trial of symptomatic patients randomizing CEA and CAS 1 to 1, had neurologic audit, and after the first 80 patients, mandated EPDs due to poor outcomes without them. The original enrollment goal was 872 patients, and although the trial was stopped after 572 patients for safety and futility reasons, the original statistical assumptions were probably overly optimistic and in error, leading to an underestimation of required subjects. However, the most significant flaw with this study is unable to be overcome: grossly underexperienced CAS operators produced excessive adverse outcomes (an almost 10% 30-day death and stroke rate), including approximately 5% emergency surgery during CAS, an event that is almost unheard of.
All but one of the investigational device exemption studies providing pivotal data for the FDA approval of carotid stents in this country were in registry format and compared their results to an objective performance goal that was an amalgam of prior outcomes in similar high-surgical-risk populations.4-6 Although admittedly not as robust as randomized trials, these studies were conducted with all of the rigor described previously, and as a result, have generally produced quality data on CAS outcomes and serve as an example of appropriate methodology.
Similarly, the multicenter studies performed after FDA approval7-9 were also conducted rigorously, and except for the absence of angiographic core laboratory oversight, have also provided high-quality data on CAS.
These registries, which generally compare outcomes in CEA to CAS, abound and unfortunately draw far-reaching conclusions that have largely gone unchallenged in spite of poor methodology and limited value data.10,11 In addition to marked comorbidity differences in the populations being studied, they are rife with selection bias and unequal neurologic audit. CAS in the United States typically requires a neurologic audit, due to its continued status as an FDA-approved but CMS-restricted technology, whereas the unrestricted practice of CEA generally does not involve a neurologist. These issues are further compounded by some analyses that include charge data, which is notoriously inaccurate, rather than direct variable costs related to the procedures. Given these inherent limitations in retrospective analyses, they are best consumed with caution. The same goes for similar retrospective analyses of stent designs, EPDs, etc.12,13
It is easy to see that given the differences in the populations treated, trial design and conduct, outcome determination, use of EPDs, operator experience, etc., that any attempt at a metaanalysis will be difficult and limited. However, that has not stopped several authors from performing them and drawing over-reaching and unqualified conclusions. One author went so far as to declare in her title that medical therapy alone was the standard treatment for asymptomatic carotid disease, without any original evidence of a differential outcome versus revascularization. The statement was based on documenting a temporal decrease in stroke rates in the medical arms of surgical trials during the past decade and a half, even though the actual medical therapy in many of those trials could not be documented, nor could achievement of target blood pressure, lipid levels, etc.14 In short, beware the meta-analysis in this field.
This article has outlined many of the necessary elements unique to CAS therapy investigation and reporting of outcomes. Although many of these elements may seem obvious or intuitive, their absence has not prevented flawed studies and conclusions from being published in high-impact journals and absorbed by a readership otherwise unfamiliar with these important factors’ influence on the results. These studies and their noncritical consumption have had a major impact on the practice of CAS, both here and abroad, and have ultimately led to limitation of access to this alternative therapy for patients at high risk for surgery, which has achieved American Heart Association guideline safety profiles, and a slowdown of technology improvements that could further enhance patient safety.
William A. Gray, MD, is Director of Endovascular Services, Center for Interventional Vascular Therapy, New York-Presbyterian Hospital/Columbia University Medical Center in New York. He has disclosed that he is a paid consultant to Pathway Medical Technologies. He has disclosed that he receives grant/research funding from Abbott Vascular, Cordis Corporation, W. L. Gore & Associates, and CREST/National Institutes of Health. He is also an owner of or shareholder in Contego Medical and Silk Road. Dr. Gray may be reached at firstname.lastname@example.org.
- Ringleb PA, Allenberg J, Brückmann H, et al. 30 day results from the SPACE trial of stent-protected angioplasty versus carotid endarterectomy in symptomatic patients: a randomised non-inferiority trial. Lancet. 2006;368:1239-1247.
- Yadav JS, Wholey MH, Kuntz RE, et al. Protected carotid-artery stenting versus endarterectomy in high-risk patients. N Engl J Med. 2004;351:1493-1501.
- Mas JL, Chatellier G, Beyssen B, et al. Endarterectomy versus stenting in patients with symptomatic severe carotid stenosis. N Engl J Med. 2006;355:1660-1671.
- White CJ, Iyer SS, Hopkins LN, et al. Carotid stenting with distal protection in high surgical risk patients: the BEACH trial 30 day results. Catheter Cardiovasc Interv. 2006;67:503-512.
- Safian RD, Bresnahan JF, Jaff MR, et al. Protected carotid stenting in high-risk patients with severe carotid artery stenosis. J Am Coll Cardiol. 2006;47:2384-2389.
- Gray WA, Hopkins LN, Yadav S, et al. Protected carotid stenting in high-surgical-risk patients: the ARCHeR results. J Vasc Surg. 2006;44:258-268. Erratum in: J Vasc Surg. 2007;45:226.
- Gray WA, Yadav JS, Verta P, et al. The CAPTURE registry: results of carotid stenting with embolic protection in the post approval setting. Catheter Cardiovasc Interv. 2007;15;69:341-348.
- Gray WA, Chaturvedi S, Verta P. Thirty-Day Outcomes for Carotid Artery Stenting in 6320 Patients From 2 Prospective, Multicenter, High-Surgical-Risk Registries. Circ Cardiovasc Intervent. 2009;2;159-166.
- Massop D, Dave R, Metzger C, et al; SAPPHIRE Worldwide Investigators. Stenting and angioplasty with protection in patients at high-risk for endarterectomy: SAPPHIRE Worldwide Registry first 2,001 patients. Catheter Cardiovasc Interv. 2009;73:129-136.
- Sidawy AN, Zwolak RM, White RA, et al; Outcomes Committee for the Society for Vascular Surgery. Risk-adjusted 30-day outcomes of carotid stenting and endarterectomy: results from the SVS Vascular Registry. J Vasc Surg. 2009;49:71-79.
- McPhee JT, Schanzer A, Messina LM, et al. Carotid artery stenting has increased rates of postprocedure stroke, death, and resource utilization than does carotid endarterectomy in the United States, 2005. J Vasc Surg. 2008;48:1442-1450.
- Iyer M, de Donato G, Deloose K, et al. The type of embolic protection device does not influence the outcome in carotid artery stenting. J Vasc Surg. 2007;46:251-256.
- Bosiers M, de Donato G, Deloose K, et al. Does free cell area influence the outcome in carotid artery stenting? Eur J Vasc Endovasc Surg. 2007;33:135-141.
- Abbott AL. Medical (nonsurgical) intervention alone is now best for prevention of stroke associated with asymptomatic severe carotid stenosis: results of a systematic review and analysis. Stroke. 2009;40:e573-583.