Even before recent shifts in insurance and health care payment systems began placing increased emphasis on comparative data, clinicians regularly pored through presentations and publications of clinical trial results in efforts to select the most effective therapies for their patients. With more trials and technologies emerging each year, understanding the strengths and shortcomings of each trial and applying it to the unique syndrome and anatomic category encountered in a given patient is more important than ever, and at times more challenging.

Although trial data may appear black and white, with similar values falling under shared headings, meaningful comparisons of these elements are rarely simple 1:1 evaluations. For example, no two trials evaluating superficial femoral artery (SFA) therapies are completely alike, with significant distinctions in how common terms like target lesion revascularization (TLR) and primary patency are defined, or how thresholds such as peak systolic velocity ratios (PSVR) vary. And, even if data from every trial were collected and evaluated in exactly the same manner, using identical criteria— a scenario that is unlikely due to the lack of consensus on many of these factors—it would be virtually impossible to conduct a variety of trials with matching patient populations, physicians with the same skill levels, and hospitals with identical equipment.

With insights from Editorial Advisory Board members and in conjunction with industry, the Endovascular Today staff has compiled a listing of current major trials evaluating nitinol stents used in the SFA, as well as ongoing trials involving drug-eluting balloons. For the reasons previously stated, these data are presented with the acknowledgement that the listings are intended as a resource, rather than a means of direct comparison. The following is a list of some, but not all, factors that should be considered in order to gain a complete understanding of a given trial’s results; these and other elements must be taken into consideration before any comparisons between trials are made.

QUALIFYING AND INTERPRETING DATA

In addition to the safety and patency data, it is important to consider the manner in which each trial defines certain terms, as these can vary in subtle yet significant ways. The severity and nature of the disease present in patients at baseline is also a considerable factor in evaluating the final results of the therapy, as these can heavily influence outcomes.

Diabetic Population

Diabetes mellitus remains one of the two most important risk factors for adverse outcomes in PAD patients who undergo revascularization. In most modern PAD device trials, one-third of patients have diabetes mellitus. This fraction increases if critical limb ischemia is included in the cohort. The higher the population of patients with diabetes, the more challenging the interventional success, and, most commonly, the worse the outcomes.

Rutherford Classification Breakdown

This is the most common classification scheme used in United States-based clinical device trials. Rutherford 1–3 represents progressively disabling intermittent claudication. Rutherford 4 describes patients with ischemic rest pain; Rutherford 5 patients have ischemic ulcerations; and Rutherford 6 patients have extensive tissue loss and gangrene. Most trials of devices for intermittent claudication include patients with Rutherford 2-4 disease, although I have always had difficulty including patients with Rutherford 4 in claudication trials. These are not the same patients as those with claudication alone. Any trial that includes patients with Rutherford 5 or 6 disease guarantees higher rates of major adverse events (MAEs), mortality, repeat interventions, and amputation.

Lesion Location

This is where “gamesmanship” comes into interpreting different vascular trials. The issue is not only the length of the lesion, but also the location of the lesion. The general rule is that more distal lesions have lower patency rates. However, with the sudden attention paid to technologies that can be used for popliteal and infrapopliteal lesions, this dogma will certainly be challenged.

Chronic Total Occlusions

Most infrainguinal peripheral arterial disease (PAD) device trials include a component of chronic total occlusions. This term highlights certain important factors. First, “chronic” suggests that the lesion is not largely thrombusbased, but more likely atherosclerotic, fibrotic, and calcific. This suggests greater challenges to therapy. Second, a total occlusion means that the artery is 100% blocked. Therefore, there are really two components to the successful intervention: crossing the lesion and then restoring patency (with meaningful durability). Both represent challenges that are not as great in lesions that are stenotic but not occluded. Trials with patient cohorts that include larger percentages of lesions that are chronically occluded will undoubtedly have lower patency rates, and may also have a higher periprocedural complication rate.

Calcification

Arterial calcification in PAD trials is common. The greater the extent of calcification, in general, the lower the patency rate and the greater the risk to the success of the intervention. The problem, predominantly in vascular trials, is that we don’t have a uniform grading scale to define the extent of arterial calcification. Most can tell if there is extensive “coral reef” calcification; however, what about mild/moderate calcification? Several investigators are working on an acceptable grading scale, which is critical to our ability to compare one trial to another.

Major Adverse Events

Investigators appreciate the importance of reporting MAEs. These may be procedure-related or after the procedure, either to 30 days or even 1 year. Recently, Conte et al1 suggested a series of adverse events which have gained enthusiasm, the most common being MALE, or Major Adverse Limb Events, defined as above-ankle amputation or the need for major reintervention. Adding mortality at 30 days to MALE is considered MALE+POD (post-operative death). The classically defined MACE (Major Adverse Cardiovascular Event) includes myocardial infarction, stroke, or death from any cause. Finally, to add durability of the intervention to the definition, one may consider RAS (reintervention, amputation, or stenosis). Readers of clinical trials must closely review the definition of MAE before comparing safety of one device to another.

Primary Patency

At first glance, this is really easy to understand. Primary patency means that you perform a procedure to restore patency to a vessel, and report how long patency is maintained without any repeat intervention. Primary-assisted patency defines the durability of an intervention that failed initially but not to the level of thrombosis and was retreated. Secondary patency means that the initial intervention failed to the level of thrombosis and was retreated. Once the second treatment was successfully performed, secondary patency defines the durability of that second intervention.

Few of us are actually statisticians, so understanding some of the different statistical strategies used make comparisons of one trial to another very challenging. I would suggest you not try. However, there is one common question I am often asked which is pretty easy, even for me to understand. What is the difference between primary patency using a Kaplan- Meier analysis or a per-protocol analysis? Kaplan-Meier analysis is a statistical tool that predicts the population in a study that survives (or reaches a certain endpoint) by a certain time. Most studies have patients who are either lost to follow-up (cannot be contacted through all efforts), choose to drop out of the study, or violate a protocol mandate. Kaplan-Meier allows for predictors of a population survival even when the population changes for reasons other than death. A per-protocol analysis means that every patient is counted, regardless of what happens to him or her. So, you can see that the two methods may come out with entirely different results.

Finally, duplex ultrasonography is the common method of measuring patency following a vascular intervention. Classically, a ratio of the fastest speed of blood flow (PSVR) within a stenosis compared to a segment of the artery proximal to the stenosis of ≥ 2 suggests > 50% stenosis, the most commonly accepted anatomic definition of patency loss. More recent data suggest that a PSVR > 2.4 is a more accurate representation of > 50% stenosis. However, this definition is less strict, and therefore, when compared to a trial with a PSVR > 2, may actually look better. Pay attention to the fine print in the “Methods” section of a manuscript.

Target Lesion Revascularization

TLR is what really matters, in my opinion, if you want to compare outcomes of vascular devices. TLR means that there was a clinical need to reintervene on the initially treated segment. This is in contrast to TVR, or target vessel revascularization, where the entire artery may have failed, but not necessarily due to the target lesion intervention. There may have been progression of native vessel atherosclerosis, for example.

Data Collection and Verification

Validity of studies can be determined by the rigor with which the trial was performed. For example, a single-center study reported by the investigator alone might provide clues as to how a certain clinical situation can be effectively managed. However, there is no validation of the data by objective, unbiased parties. An independently adjudicated study carries far greater credibility when determining the validity of a conclusion. The more independent the data collection and verification, the more credible the conclusions.

Stent Fracture Rates

The rates of stent fracture reported in the various trials can be misleading. Some are core lab adjudicated, while others have just been described by the treating physician. Some core labs have a great deal of experience looking at radiographs with stent fractures, and others do not. And finally, the fracture rates that are presented in trials depend on the denominator used to calculate the rate. Some studies divide the number of fractures by the total patient population, some by the total number of stents used, and others by the number of patients or stents that had films that could be evaluated by the core lab. Each method gives you a different value, so again the stent fracture rates may not be comparable.

Randomized Versus Single-Arm/Registry

The “gold standard” trial design is one in which multiple centers participate in a randomized trial of two (or more) treatments in the same patient population. This removes bias of one treatment only by all centers. However, this trial design is by far the most complex, most expensive, and takes the longest to complete. One nuance of randomized trials is how they count a patient who was randomized to treatment A (let’s say optimal medical therapy), but crosses over (for a whole host of reasons) to treatment B (the actual endovascular intervention). This raises the concept of on-treatment versus intention-to-treat analysis. On-treatment means whatever treatment the patient actually received versus intention-to-treat, which means that the patient is analyzed based on treatment initially assigned. In short, this means that if a patient is initially randomized to treatment A, but ultimately received treatment B, in the intention-to-treat analysis, the patient is included in treatment A. In this scenario, the on-treatment analysis means that the patient would be included in treatment B. Therefore, results would be different depending on how results are analyzed and reported.

Prospective Versus Retrospective

A prospective trial means that you choose time A, and after that, begin enrolling patients into a trial. This is much more scientifically valid than retrospective, where data is determined by looking back at how patients were treated for a certain problem. Retrospective trials include significant bias and are used to raise questions that may form the basis of subsequent prospective trials.

Investigator/Site/Region Influence

An important but often overlooked component of a trial is the skill and experience of the investigator. When a new treatment is being studied, one would expect that the investigators have the greatest knowledge and experience in the use of a specific technology, and would suggest that results may not translate as well when less experienced operators are included. This is not always the case, but it is a generally accepted rule. As devices become easier to use, this factor becomes less important.

Industry Influence

This is the reality of the world—most new vascular device trials are sponsored, in some way, by the company developing the device. There is absolutely nothing wrong with this. However, the actual trial design may result in criticism. Data collection on a device with only self-reporting of data by an investigator does not carry much validity and will not necessarily expand the body of scientific literature to aid clinicians in determining optimal treatment strategies for their patients. However, one may clearly understand the hesitancy of corporations to support prospective, multicenter, randomized trials. As previously mentioned, this takes longer, costs more, and is clearly riskier than prospective, single-arm registries. Put yourself in their shoes—would you want to put your shareholders at this level of risk? So, as long as industry-sponsored trials are run ethically with independent (noncorporate employees; all disclosures publicly available) adjudication of all endpoints, the data has validity and is important.

Michael R. Jaff, DO, is Professor of Medicine, Harvard Medical School, and Chair of the MGH Institute for Heart, Vascular and Stroke Care, Massachusetts General Hospital in Boston. He has disclosed that he is a Board Member, VIVA Physicians, a 501(c)(3) not-for-profit education and research organization; a non-compensated advisor for Abbott Vascular, Boston Scientific, Cordis Corporation, Covidien Vascular, and Medtronic Vascular; and has equity Investment in PQ Bypass. Dr. Jaff may be reached at (617) 726-3784; mjaff@partners.org.

  1. Conte MS, Geraghty PJ, Bradbury AW, et al. Suggested objective performance goals and clinical trial design for evaluating catheter-based treatment of critical limb ischemia. J Vasc Surg. 2009;50:1462-1473.