Evaluating Outcomes in Context
Robert Zwolak, MD, discusses his work with the SVS Outcomes Committee, the role of AHRQ, and his perspective on contemporary endovascular aneurysm trials.
Endovascular Today: What is the interest of the SVS Outcomes Committee regarding endovascular aneurysm repair?
Dr. Zwolak: The SVS Outcomes Committee, under Greg Sicard, MD, has become very active in the past year in terms of analyzing the results of the major procedures that vascular surgeons perform. We have been heavily involved in carotid stenting as well as endovascular aneurysm repair. The interest in endovascular repair was primarily stimulated by the marked publicity surrounding EVAR-1, EVAR-2, and DREAM, but particularly EVAR-2, in which the perioperative and all-cause mortalities seemed to be very substantial. Dr. Sicard and other members of our committee predicted that the results would be substantially different if the trial had been performed in the US. We also realized that the US Agency for Healthcare Research and Quality (AHRQ) had decided to look at the data surrounding endovascular aneurysm repair, and they seemed to be heavily focused on the results of the EVAR-2 trial. It was incumbent on us to look at the US data because we believed that it would probably be different.
Endovascular Today: What is AHRQ and how you think these data will affect the concerns of AHRQ?
Dr. Zwolak: AHRQ is a relatively new arm of the government, physically situated in Rockville, Maryland, and Dr. Carolyn Clancy, is the Director. It is a group of medical scientists and epidemiologists with a charge of improving quality, safety, efficiency, and effectiveness of healthcare for all Americans. The SVS first became aware of AHRQ several years ago when we learned that AHRQ was going to study the issue of screening of abdominal aortic aneurysms (AAAs). Throughout 2003 and 2004, we made several trips to AHRQ to discuss their AAA screening project. At the end of 2004, we were consulted briefly regarding their draft document on AAA screening. We held several conference calls expressing concern regarding certain aspects of their manuscript. Their document was published in January 2005. I suspect their review of AAA screening may have been the source of their interest in endovascular aneurysm repair, but it was my impression that the analysis may have been commissioned by an outside agency or group.
Endovascular Today: Do the data that you presented at the SVS meeting appropriately address the concerns of AHRQ as far as endovascular repair is concerned?
Dr. Zwolak: We will not know until we see their highly anticipated, unreleased document. We hope that AHRQ will look at the US data more closely and consider it more reflective of what is going on in the US than do the EVAR-2 data. On the one hand, AHRQ looks at levels of evidence, and EVAR-1, EVAR-2, and DREAM were prospective, randomized, controlled trialsÑthe highest, purest science. In contrast, the SVS Outcomes Committee evaluation of the IDE trials was a subset analysis of previously collected data, therefore a lower level of scientific data.
Each of those types of study (retrospective versus prospective, randomized) has its benefits and detriments. For instance, in the randomized, controlled trials (EVAR-1 and EVAR-2) close examination of the data reveals that a fair number of patients in both groups died before they even received treatment. Using intention-to-treat analysis, if a patient is randomized to endovascular repair or open surgical repair and dies before their procedure, that death gets assigned as to the particular treatment arm. Because many patients in EVAR-2 died preoperatively, the outcome of endovascular repair appeared less efficacious. You can argue that this is pure science, but in real life, preoperative deaths are not considered part of postoperative mortality. On the other hand, with the retrospective analysis of the IDE trial, it can be said that the trial was not designed to look at high-risk patients and that a post-hoc analysis is impure. But if there ever were an example of a highly scrutinized, highly audited, super-compliant dataset in the US, it would be data collected for device approval under the auspices of an FDA IDE trial. Thus, each of the two approaches has pros and cons, but in the view of AHRQ, the randomized controlled trials represent the most sophisticated scientific analysis.
Endovascular Today: Do you think that data from a trial conducted in a particular country can be effectively used to determine payment/reimbursement policy in another country?
Dr. Zwolak: This is a complex question. One argument is that data from the US should be the gold standard for payment and coverage decisions in the US. The people at AHRQ might answer by saying that a randomized, controlled trial performed in the UK using devices similar or equivalent to those available in the US has a higher value than the EVAR studies done in the US. I suspect AHRQ might criticize American medical scientists for not having run randomized trials, but the FDA and study investigators believed the questions of safety and efficacy could be answered accurately without the need for randomization and, in my opinion, they were right. I believe that the very best analysis would stem from objective review of all available data, with equal scrutiny to the strengths and weaknesses of both approaches.
Endovascular Today: What are your thoughts regarding DREAM, EVAR-1, and EVAR-2 and their focus on all-cause mortality?
Dr. Zwolak: All-cause mortality isÊan extremelyÊimportant variable, but to expect that a disease-specific treatmentÑAAA repairÑshould have a major, long-lasting impact on all-cause mortality in a senior citizen population is unrealistic. For instance, 48 patients in EVAR-1 died from cancer. This was about one-third of the overall mortality throughout the duration of the study, and these cancer deaths diluted the overall survival benefit conferred by endovascular aneurysm repair over open repair. Should we somehow expect that AAA treatment should reduce cancer deaths?
My point is that all-cause mortality is a metric of limited value when assessing a disease-specific treatment in the senior citizen population.
Endovascular Today: What do you think is the major take-home message regarding endovascular aneurysm repair in high-risk patients?
Dr. Zwolak: I think that these trials, like any science, need to be read in substantial detail, and it is dangerous to take a one-liner home from a trial as complex as EVAR-2. EVAR-2 is important because it shows that patients with aneurysm disease who have advanced comorbidities have a distinctly limited life expectancy. The doctor and patient must decide on an individual basis whether any aneurysm repair should be undertaken, or as EVAR-2 might suggest, whether the patient is so ill that treating the aneurysm would not confer any survival benefit.
The EVAR-2 authors' conclusions about endovascular aneurysm repair in high-risk patients were very pessimistic based on their high perioperative mortality (much of that was preoperative mortality), and a very poor 4-year survival rate. The US high-risk patients demonstrated much lower 30-day mortality and substantially better 4-year survival. EVAR-2 makes one believe there is a set of patients who are too sick to undergo endovascular AAA repair. The US data leads me to the conclusion that the "too sick for EVAR" cohort represents an extremely small subset, and that these few patients are well beyond what we would typically call "high risk." So, while most high risk patients will survive EVAR and receive excellent protection from AAA-related death, our task now is to identify the characteristics of that tiny subset of patients who are "too high risk."
Robert Zwolak, MD, is a vascular surgeon at Dartmouth-Hitchcock Medical Center, Section of Vascular Surgery, Lebanon, New Hampshire, and Chair of the National Aneurysm Alliance. He may be reached at (603) 650-4682; email@example.com.