Carotid Artery Stenting: Does Experience Matter?
An analysis of published series underlines the importance of learning curve and experience.
Carotid intervention, be it by endovascular or surgical means, has prophylactic intent—the ultimate aim is survival free of ipsilateral stroke, but the longer-term benefit that might be achieved must be weighed against the "up-front" risk. In general, the risk-benefit ratio has narrow margins particularly for asymptomatic patients.
Level 1 and supporting evidence from Europe and the US indicates a linear relationship between throughput and outcome for carotid endarterectomy (CEA).1,2 The relationship between throughput and outcome for carotid artery stenting (CAS) is less clear. However, as a highly technically complex procedure, it is likely that such a relationship exists. In this article, we present the motion that for CAS, both workload and experience are likely to have an impact on procedural outcome and that CAS, like any complex intervention, may well be unsafe in low-volume centers.
SOURCES OF DATA AND THEIR LIMITATIONS
Gathering outcome data for CAS is complicated by a number of issues. Specialists from a number of backgrounds perform CAS, including interventional radiologists, interventional neuroradiologists, interventional cardiologists, vascular surgeons, neurosurgeons, angiologists, and interventional neurologists. Each specialty brings its own unique perspective, but these perspectives are not necessarily comparable, and different specialists may arguably start at different points of the learning curve.
Second, in appraising the outcome of CAS, it can be difficult to extract the influence of the individual/center learning curve from the influence of technical advances (and "world learning") that have occurred with time. Notable developments include dedicated carotid stents, fine guidewire technology, and improved understanding of the importance of periprocedural hemodynamics and pharmacological support. There are robust data (level 1 evidence for the dual-antiplatelet regimen)3 that many of these innovations have a significant impact on outcome, quite apart from the influence of the learning curve.
Third, with respect to outcomes for CAS, it would appear that although superficially there are many sources of data, which include randomized controlled trials (RCTs), registries, and device-related postmarketing surveillance (PMS) studies, these are generally opaque to public scrutiny regarding the relationship between experience and volume and outcome. RCTs are either proscriptive (physicians performing CAS within CREST, for example, have to meet certain stringent criteria),4 or, for pragmatic reasons, after initially attempting to control standards within trials, steering committees may allow less-experienced interventionists to participate, mindful of the otherwise painfully slow recruitment rate (an argument levelled against the EVA-3S trial).5 Neither situation allows a realistic analysis of the volume-outcome or experience-outcome relationships.
Reporting of results outside of RCTs has mainly been performed on a voluntary basis. Submission of outcome data to national registries is generally voluntary, and the reported outcomes are mostly self-audited. The limitations of self-auditing need hardly be stated.6 Recent legislative demands for PMS studies in the US have resulted in a surge in data collection, and these registries are designed such that their data are applicable to real-world situations. However, participation is inevitably influenced by industry, and invitation to participate in such PMS studies is generally based on a center's general experience with CAS and its specific experience with the carotid stent and protection device being evaluated. The relationship among experience, volume, and outcome cannot easily be teased out of such an arrangement. In all the given examples, it is not known how many CAS procedures have been performed outside of the trials/registries by participating physicians, and so their overall numbers are not easily calculable. It is likely, also, that publication bias will have played a role. Centers or individuals with poor results are less likely to have published their data.
Finally, data are often entered into both RCTs and national registries (and possibly, PMS studies) as a matter of good practice; hence, data duplication and "redundant publication" is a genuine problem when pooling results.
RANDOMIZED CONTROLLED TRIALS
To date, eight RCTs of CAS versus surgical endarterectomy have been performed; six had more than 100 patients, randomizing a total of 2,888 patients.5,7-11 Two of these six trials offered some insight into the influence of learning curve. Within CAVATAS, comparing carotid angioplasty and CEA, data from individual centers were analyzed. The investigators stated in response to a letter regarding the trial that, "Increasing experience, better technology, or both clearly made an important difference to the safety of endovascular treatment. The rate of stroke at the two centers with the greatest experience dropped from a mean of 11% in the first 50 patients assigned endovascular treatment to 4% in patients treated subsequently. The rate of stroke seemed to be higher in the smaller centers, but because many joined the trial toward the end of recruitment, an effect on the possible learning curve does not appear in the overall analysis over time."12
EVA-3S offered data regarding case volume and outcome in 260 patients in whom CAS was attempted.5 Recruiting centers were a mixture of academic and nonacademic (totaling 20 and 10, respectively). No differences in outcome were seen in centers that enrolled <21, 21 to 40, or >40 patients. When comparing the enrolling physician's experience, no statistical difference was seen among those with experience (>50 procedures), those with less than a 50-procedure experience, and those still being proctored within the trial; the major adverse event (MAE) rates were 12.2%, 11%, and 7.1%, respectively.
The endovascular technique was rudimentary in CAVATAS (this trial largely predated "contemporary practice" with only 26% of patients undergoing stent placement—largely for "bail-out"), and neither it nor EVA-3S were, of course, powered to make a meaningful comparison between experienced and inexperienced physicians performing CAS—this would have required a randomized comparison in its own right. For both CAVATAS and EVA-3S, the differences in outcomes were not statistically significant, and the risk of a type II error due to relatively small numbers cannot be overlooked. Furthermore, a staggering 85% of operators performing CAS within EVA-3S had performed fewer than 50 procedures in total,13 which is generally considered to represent intervention performed on the steep part of the operator learning curve.14,15
INDEPENDENT CAS REGISTRIES
Although registry data are generally less reliable than RCT data, advantages include the fact that registries are less proscriptive regarding inclusion/exclusion criteria and types of stent and protection device used. They might, therefore, be expected to have "captured" a larger proportion of the eligible cases.
The largest of the independent CAS registries is the Global Carotid Artery Stent Registry. The 2003 update included data from 53 centers and included a total of 12,392 procedures worldwide.16 The article graphically showed a steep learning curve for CAS (performed with distal protection). Stoke and death rates for centers that had performed 20 to 50 procedures were 4.04% compared with 1.56% for those that have performed more than 500. However, it is unclear from these data what the minimum number requirement for improved outcomes is (presumably somewhere between 50 and 500).
The German Arbeitsgemeinschaft Leitende Kardiologische Krankenhausärzte (ALKK) included the results from 1,888 patients treated in 28 hospitals during a 9-year period.17 Analysis of these data confirmed a progressive reduction in stroke rates from 1996 to 2004 (6.3% to 1.9% respectively; P<.02). Another sizeable German registry, the Pro-CAS, reported the combined rate of permanent neurological deficit and death in three cohorts: 735 patients treated before October 2000, 923 patients undergoing unprotected CAS between October 2000 and 2003, and 1,609 concurrent patients undergoing protected CAS between October 2000 and 2003.18 The respective event rates were 4.6%, 2.2%, and 2.1%. Although some of the improvement in outcomes between the pre-2000 and post-2000 cohorts may be ascribed to certain technological advances, namely dedicated carotid stents,19 it is difficult to escape the conclusion that experience has an important influence on outcome, particularly as the adoption of protection devices after 2000 did not lead to a significant improvement in results.
POSTMARKETING STUDIES AND REGISTRIES
With the intention of showing safety in the real world, PMS studies are increasingly used as a condition of approval for individual stent and protection systems by bodies such as the FDA, and approval is linked to specific stent/protection system packages. Many PMS registries recruit physicians according to their level of experience. It is difficult, therefore, to evaluate the effect of a learning curve or the influence of volume on outcome.
It is notable that two registries, CASES-PMS and CAPTURE, sought to explore the influence of a structured training program on procedural event rate.20,21 Proponents of the "experience does not matter" theory would use these data to support their argument; both registries showed little correlation between volume/experience and outcome. However, this is likely what the industry behind the Precise stent/AngioGuard filter (CASES-PMS) (Cordis Corporation, Warren, NJ) and the Acculink stent/Accunet filter (CAPTURE) (Abbott Vascular, Santa Clara, CA, formerly Guidant Corporation) registries wished to show—that it is not beyond the reach of relatively inexperienced physicians to perform CAS safely with their equipment. What these data actually show is that you can take relatively inexperienced physicians and expose them to a structured CAS training program and get them up to speed without prohibitive difficulty. The structured training program is the key. Without it, these inexperienced physicians and the industry supporting them may well have found that many more procedures would have had to be performed before the "trial" physicians were in any way comparable to their more experienced counterparts.
EVALUATION OF THE LEARNING CURVE
Perhaps the most effective way of analyzing the influence of the learning curve is to evaluate outcomes in single centers over time, after technical advances in equipment have been made (ie, after around 2001) and after the physicians performing CAS have familiarized themselves with cerebral protection devices if these devices were to be part of their routine practice.
Early single-center data come from Roubin et al from 1994 to 1999; this entire period predated cerebral protection devices.22 On an annual basis, the incidence of minor stroke declined from 6.8% (1994–1995), to 5.8% (1995–1996), 5.3% (1996–1997), and then 4% (1997–1998), with no major strokes or neurologic deaths occurring during 1997 to 1998.
Ahmadi et al evaluated 320 CAS procedures as four groups of 80 cases, although this study predated many important technological advances in CAS. There was a significant reduction in the frequency of neurological complications after the initial 80 interventions (P<.03), but technical success was not appreciably improved with increasing experience thereafter. It was concluded that a relatively large number of interventions (ie, 80) should be performed to overcome the negative effects of the initial learning phase.14
Lin et al presented their results in 200 consecutive CAS procedures in 182 patients in a more contemporary time frame. The results were analyzed in four sequential groups of 50 procedures. The 30-day stroke and death rate was 8% in the first cohort, 2% in the second, and zero in cohorts three and four. Their article suggests a learning curve of 50 procedures.23
Single-center data have also recently been published by Verzini et al on the learning curve for CAS.15 Data were published with both yearly outcomes from 2001 to 2006, plus the number of procedures. A total of 627 protected CAS procedures were performed between 2001 and 2006. When comparing the results of the first 3 years (n=195 CAS procedures) with the second 3 years (n=432 CAS procedures), the 30-day major stroke and death rate decreased from 3.1% to 0.9% (P<.047), and the 30-day any stroke and death rate decreased from 8.2% to 2.7% (P<.005). The investigators concluded that the results highlighted the importance "of an appropriate learning curve that involves a caseload larger than that generally accepted for credentialing."
The available data support the perception that results for CAS have improved with time. The influence of advances in technology on the reduction in adverse event rates from CAS is hard to separate from the influence of individual learning curves. Due to the overlapping nature of registries and trials, the limitations of self-audit affecting the former and the proscriptive/restrictive nature of the latter, it is difficult to soundly pool the limited available data. Perhaps the most convincing data (particularly when graphically displayed) come from single-center experiences with stable (ie, nonevolving) CAS technique, many of which show clear reductions in event rate with passing years.
"By three methods we may learn wisdom: first, by reflection, which is noblest; second by imitation, which is easiest; and third by experience, which is the most bitter."
—Confucius, 551–479 BC
Sumaira Macdonald, MBChB (Comm.), FRCP, FRCR, PhD, of the Steering and Technical Management Committees of the Asymptomatic Carotid Surgery Trial (ACST-2), is a Consultant Vascular Radiologist and Honorary Clinical Senior Lecturer at Freeman Hospital, Newcastle upon Tyne, England. She is a collaborator and proctor (physician instructor) in the International Carotid Stenting Study (ICSS) and sits on the Steering and Technical Management Committees of the ACST-2 Trial. She has received research grants from Abbott Vascular, Cordis, Johnson & Johnson, Gore, and Medtronic for research into carotid stenting and cerebral protection. Dr. Macdonald may be reached at firstname.lastname@example.org.
Jonathan Smout, MBChB, MD, FRCS, is Specialist Vascular Registrar, Northern Vascular Center at Freeman Hospital, Newcastle upon Tyne, England. He had disclosed that he holds no financial interest in any product or manufacturer mentioned herein. Dr. Smout may be reached at +0191 244 8434; email@example.com.
Gerry Stansby, BA, MBBChir, MA, FRCS, MChir, is Professor of Vascular Surgery, Northern Vascular Center at Freeman Hospital, Newcastle upon Tyne, England. He has disclosed that he holds no financial interest in any product or manufacturer mentioned herein. Dr. Stansby may be reached at +0191 223 6161 ext. 37104; firstname.lastname@example.org.