SPEED: A New Initiative in Real-World PAD Evidence Evaluation
An overview of the FDA's new multistakeholder project to support real-world evidence evaluation for devices aimed at treating peripheral artery disease.
The FDA recognizes the need to bring innovative medical products to patients in the United States in a cost-effective and timely manner to promote public health. As such, FDA’s Center for Devices and Radiological Health (CDRH) supports the contribution of real-world evidence (RWE) to streamline the regulatory process, when appropriate, through the generation of robust and meaningful evidence to support the safety and effectiveness of devices. RWE is defined as clinical evidence regarding the usage and potential benefits or risks of a medical product derived from an analysis of real-world data, such as registries, electronic medical records (EMRs), claims data, and other sources.1 To promote collaboration and encourage the collection of high-quality RWE, CDRH has participated in the establishment of the Medical Device Epidemiology Network (MDEpiNet), a public-private partnership to advance the nation’s approaches to the evaluation of medical devices, and the National Evaluation System for Health Technology Coordinating Center.
From these efforts, the MDEpiNet Registry Assessment of Peripheral Interventional Devices (RAPID) project emerged from the Predictable and Sustainable Implementation of National Registries for Cardiovascular Devices program.2 Under the project management of the Duke Clinical Research Institute, the RAPID project was designed to advance the foundational elements of a total product life cycle approach for the evaluation of medical devices used to treat and manage peripheral artery disease (PAD) through a multistakeholder collaboration between clinicians, medical professional societies, patient advocacy groups, United States federal agencies, health information technology vendors, clinical research organizations, and medical device manufacturers. Use of RWE in the peripheral vascular space offers an opportunity to overcome challenges traditionally associated with evaluating peripheral vascular interventions (PVIs) owing to the heterogeneity of the disease, variability in comorbidities, availability of numerous device types and treatment strategies, and the lack of consensus among medical specialties on the best treatment strategy for a given patient population.
The first objective of the RAPID program defined a minimal set of core data elements related to interventional treatments of patients with PAD. Clinicians, industry, professional societies, and regulators collaborated to determine the minimum set of information most important for clinical and regulatory decision-making.3,4 The elements were first incorporated into the Society for Vascular Surgery’s Vascular Quality Initiative (VQI) registry, with the goal of integration into multiple registries and health care information systems to support a consensus lexicon and promote interoperability across clinical sites, EMRs, and national and international registries. These standard data elements are intended to be used by all RAPID stakeholders in conjunction with data elements from the Global Unique Device Identification Database (GUDID) to create a structured data set that supports pre- and postmarket assessment, quality improvement, and safety surveillance of peripheral interventional devices across registries and clinical sites.
For the second phase, the RAPID partners proposed the SFA-Popliteal Evidence Development (SPEED) project to demonstrate the value of integrating standardized core data elements and establish a methodology to use RWE to support clinical and regulatory decision-making in a complex and evolving patient population. Using data extracted from the VQI PVI registry, the SPEED project will generate contemporary objective performance goals (OPGs) to be used as comparators for evaluating interventions in the superficial femoral and popliteal arteries. The OPGs have the potential to be used to support clinical evaluation of medical devices or expand the approved indications for marketed devices. The SPEED OPGs will include multiple key outcomes and will describe statistically significant covariates based on patient characteristics and disease severity using multivariate analysis. The intention is to repeat these analyses over time, as other registry data sources are available, to reflect contemporary practice and keep pace with evolving technologies and procedures.
CREATING A ROADMAP
One goal of the SPEED project was to assess the feasibility and utility of both the process and the OPG outputs to provide a meaningful framework for future projects and identify specific limitations and factors when using RWE to evaluate treatment of femoropopliteal disease. The SPEED development process identified obstacles that needed to be addressed to ultimately develop OPGs from real-world registry data. These hurdles included funding, informatics challenges, legal considerations, data transfer and governance, integration of regulatory agencies and processes, and incorporation of industry and academic expertise. By forming multiple working groups and interacting with experienced external partners, the RAPID team managed to insource and identify viable paths to overcome each challenge efficiently in parallel. For example, while an informatics working group focused on incorporating unique device identifiers on device labels, enhancing GUDID, and supporting interoperability of the core elements across multiple sources, the governance working group aimed to develop a roadmap to transfer deidentified patient-level data from an agnostic source (eg, VQI) to a compliant analytic environment to ensure patient protection measures.
The RAPID protocol working group collaborated extensively to determine important endpoints, definitions, and covariates for clinical and regulatory purposes with support from VQI to identify the extent and quality of information captured in their registry. During the OPG development process, several limitations and challenges were identified, including discordance between the type of data captured in typical clinical use and those used for regulatory assessment of device performance (eg, vascular patency), reliability of operator-entered real-world data, attribution and adjudication of events in cases of multiple treatments and lesions, and specific device identification since the evaluable data were pulled prior to Unique Device Identification (UDI) availability. The group developed a prospective statistical analysis plan to evaluate deidentified, patient-level data captured in the VQI PVI registry according to specific inclusion and exclusion criteria. Although the analyses are currently ongoing, the group has proposed publishing the methodology and OPGs for technical procedural success and 1-year outcomes, including mortality, major amputation, amputation-free survival, target lesion revascularization, and target lesion occlusion, stratified by common treatment types (eg, angioplasty, mechanical atherectomy, stents, and combinations thereof). Interested sponsors will therefore have the ability to use the same approach to perform their own customized analyses based on their particular needs.
The ultimate goal for SPEED is to demonstrate the measurable utility of the SPEED OPGs to support efficient device evaluation and regulatory decision-making. Once functionality has been established, it is expected that these contemporary performance goals may replace existing performance goals (eg, de novo stenting) to support expansion of conventional indications where the potential benefits and risks are well known. Once robustness and confidence in the methodology and data are further ascertained, it is anticipated that the application of the data may be expanded to include more novel regulatory applications. For example, for indication expansions with greater uncertainty (ie, due to the device design, technique, or other factors), these real-world data may be valuable for comparing strategies, such as propensity matching using patient-level data captured from the registry or other data sources, that may offer time- and cost-saving opportunities over conventional clinical studies. Over time, the methodology could also be extended to develop OPGs in other challenging areas, such as below-the-knee peripheral interventions due to similar challenges associated with femoropopliteal disease (eg, disease heterogeneity, numerous devices, techniques, and clinical specialties), a lack of clear correlation between vessel patency and other clinical measures, and inconsistent results in literature. In the meantime, the OPGs may also be valuable to inform clinical trial design (eg, generate hypotheses, determine sample size), particularly for trials intended to support specific indications tailored to patient and lesion characteristics where data from the literature may be inconsistent or absent.
As the feasibility and utility of the OPGs continue to be established, we recognize that the OPGs developed from real-world use may provide less precise estimates of benefit and risk given the passive collection of clinical endpoints as compared to a protocol-driven trial. Conversely, we also recognize that, although protocol-driven clinical trials provide more control and precision, they also may not reflect actual clinical use and outcomes of a marketed device. Here, the SPEED project is considered valuable in establishing the initial viability of an OPG development process with the awareness that ongoing iterations and improvements will be necessary to address and mitigate the limitations of RWE use.
Although RWE can contribute to a fuller understanding of the benefits and risks of medical devices and procedures in a real-world patient population in an efficient and cost-effective manner, there are multiple factors to consider when using RWE to support clinical and regulatory decision-making. Because there are potential limitations in endpoint availability, data relevance, quality, relatability, and bias, it is important to understand how these limitations may impact the overall utility of RWE and attempt to minimize the limitations whenever possible. Other factors that should be considered when determining the applicability of RWE to support regulatory decision-making include the historical experience with the device and indications, the quantity and quality of the data, known and unknown covariates that may not be adequately captured, and the value of active versus passive collection of clinical endpoints.5
The SPEED project paves the way forward to understand and overcome these limitations by creating a roadmap for future initiatives. The RAPID group hopes to continue building on these efforts to improve the infrastructure for real-world data collection and quality to more effectively inform clinical and regulatory decisions. This infrastructure and methodology generated for the RAPID project can be leveraged for future projects to advance the quality, reliability, and use of RWE to support clinical, regulatory, and marketing decisions across the total life cycle of a product. The FDA believes that clinical societies, registries, and EMR organizations should continue to collaborate to incorporate other minimal core data elements and build coordinated registry networks with an aligned lexicon for patient information and clinical outcomes to generate multisource data sets to support future clinical studies and analyses. We also recognize that the continued success of projects like RAPID and SPEED is dependent on participation and adoption by individual physicians. Incorporation of a common lexicon in conjunction with UDI in EMRs and registries promotes consistency across data collection technologies and improves efficiencies for health care providers, particularly at the initial point of patient care by improving autocapture of important device information to bolster the clinical workflow. By improving clinical workflow and data capture, the quality and reliability of RWE should increase, which in turn will contribute to higher-quality care and regulatory decision-making.
1. US Food and Drug Administration. Use of real-world evidence to support regulatory decision-making for medical devices: guidance for industry and food and drug administration staff. https://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm513027.pdf. Published August 31, 2017. Accessed July 20, 2018.
2. Morales JP, Cronenwett J, Thatcher R. Registry assessment of peripheral interventional devices (RAPID). Endovasc Today. 2016;15:85-94.
3. Jones WS, Krucoff MW, Morales P, et al. Registry assessment of peripheral interventional devices (RAPID): registry assessment of peripheral interventional devices core data elements. J Vasc Surg. 2018;67:637-644.
4. Jones WS, Krucoff MW, Morales P, et al. Registry assessment of peripheral interventional devices (RAPID): registry assessment of peripheral interventional devices core data elements. Circ J. 2018;82:316-322.
5. Whatley E, Malone M. Current considerations on real-world evidence use in FDA regulatory submissions. Endovasc Today. 2017;16:106-108.
Misti Malone, PhD
US Food and Drug Administration
Center for Devices and Radiological Health
Office of Device Evaluation
Division of Cardiovascular Devices
Peripheral Interventional Devices Branch
Silver Spring, Maryland