The Value of Propensity-Matched Registry Data for New Device Evaluation

FDA disclaimer: The mention of commercial products, their sources, or their use in connection with material reported here is not to be construed as either an actual or implied endorsement of such products by the Department of Health and Human Services.

Many medical device clinical trials, especially for lower-risk class II devices, are conducted as single-arm studies rather than randomized controlled trials. Identifying an appropriate comparator for the devices in these trials is challenging because historical publications may not reflect contemporary practice, may not account for potential confounders, and do not provide patient-level data for accurate matching of patient populations. This can result in an inappropriately generated performance goal (PG) and a potentially inconclusive outcome for the study.

Use of Medical Registry Data to Supplement New Device Trials

Medical registries provide a potential source of real-world evidence (RWE) to use as a contemporary comparator in a new device trial if required conditions are met.¹ These include sufficiently granular clinical data, including device identifiers, capture of appropriate endpoints, and adequate follow-up. The Society for Vascular Surgery Vascular Quality Initiative (VQI) registry has been used in numerous regulatory applications, including to support postapproval requirements, indication expansions, and to provide comparator data for a new device.^2,3 VQI data have recently been used to create objective PGs (OPGs) for balloon angioplasty, stent, and atherectomy treatment in the SPEED study.⁴ The SPEED analysis pointed out that global OPGs provide useful device comparisons but that patient or disease characteristics of a specific device trial might differ from a global cohort and thus benefit from a matched subset of patients to provide the most accurate OPG.

A potential challenge in using medical registry data to provide a contemporary control group is that core lab measurements of radiographic images are often needed for key outcome analysis, but images are not usually available in medical registries. However, it is possible to use existing clinical data from the registry to identify an appropriate control group, such as with propensity score matching of key clinical variables, and then obtain radiographic images from participating sites for these specific cases for core lab analysis. This represents an efficient way to focus image retrieval on a preidentified control group while avoiding the impractical capture of images for all patients in a registry.

USING REGISTRY-BASED PROPENSITY SCORE MATCHING FOR NEW DEVICE EVALUATION

A recent example illustrates the value of a registry-based, propensity score–matched control group with core lab imaging for key outcomes. A single-arm trial of a new atherectomy device (FreedomFlow orbital circumferential atherectomy system, Cardio Flow, Inc.) was performed using a comparison with published outcomes of previously cleared atherectomy devices. However, it was determined that when compared with the original prior atherectomy trials, the contemporary Cardio Flow trial enrolled patients with more critical limb ischemia and longer lesions, which portended worse outcomes. This suggested that the literature-derived historical PG may have been overly conservative for assessing the performance of the FreedomFlow system.

To remedy this discrepancy, Cardio Flow, the FDA, and the VQI collaborated to develop a prospectively designed RWE evaluation of the FAST II data using a propensity-matched registry comparator. Patients treated with commercially available rotational or orbital atherectomy devices during the same time interval and using the same inclusion/exclusion criteria as FAST II were selected from the VQI registry. Propensity score analysis showed a good match of key patient and lesion characteristics between FAST II and VQI patients. Angiographic images for the matched cases were then obtained from VQI centers and deidentified by an independent group (Fivos Health) before random submission with FAST II cases for blinded core lab measurement of key outcomes by NAMSA. Core lab measurements were then transmitted to an independent statistician who compared the FAST II and matched VQI control treatment outcomes using the prespecified statistical analysis plan.

This method resulted in FAST II and control cohorts that were well matched for key clinical variables, including disease severity and lesion length, with mean baseline diameter stenosis for FAST II of 62% compared to 62% for VQI registry control cases. The mean postatherectomy diameter stenosis for FAST II was 41% compared to 46% for peripheral vascular intervention. This satisfied the statistical criterion to demonstrate noninferiority of the FAST II treatment compared to contemporary treatment with commercially available atherectomy devices in VQI.

CONCLUSION

This novel use of RWE from VQI, including patient-level data and core lab analysis of original angiographic images, illustrates the value of using RWE to create a propensity score–matched control group for new medical device evaluation, especially when the limitations of a single-arm trial may not allow for an appropriate evaluation. The ability to obtain original radiographic images from prior cases in VQI for core lab analysis expedited the timeline for the creation of a robust control group that contained the level of evidence FDA recommends in a prospective study. This allowed more efficient clearance of a new atherectomy device that would likely not have been possible using historical literature-derived comparators and underscores the value of RWE to meet regulatory requirements.

1. US Food and Drug Administration. Guidance document: draft: use of real-world evidence to support regulatory decision-making for medical devices. December 2023. Accessed February 9, 2024. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/draft-use-real-world-evidence-support-regulatory-decision-making-medical-devices

2. Whatley E, Malone M. Current considerations on real-world evidence use in FDA regulatory submissions. Endovasc Today. 2017;16:106-108.

3. US Food and Drug Administration. Examples of real-world evidence (RWE) used in medical device regulatory decisions. Accessed November 28, 2023. https://www.fda.gov/media/146258/download

4. Bertges DJ, White R, Cheng YC, et al. Registry Assessment of Peripheral Interventional Devices objective performance goals for superficial femoral and popliteal artery peripheral vascular interventions. J Vasc Surg. 2021;73:1702-1714. doi: 10.1016/j.jvs.2020.09.030