Current Landscape for Global Regulatory Acceptance of Real-World Evidence for Medical Devices

Although traditional protocol-driven clinical studies play an important role in evaluating novel medical devices, real-world evidence (RWE) is also valuable for understanding device safety and effectiveness under broader conditions of use and with potentially less commitment of time and resources. Because of these potential benefits, there is ongoing interest in developing methodologies suitable for leveraging RWE that are relevant and reliable for regulatory uses.

The Center for Devices and Radiological Health (CDRH) of the US Food and Drug Administration (FDA) has included such efforts in recent strategic plans and published guidance on RWE use in 2017.¹ In addition, since its initiation in 2015, CDRH has actively participated in the Registry Assessment of Peripheral Interventional Devices (RAPID), a multi-stakeholder collaboration involving clinicians, academia, regulators, and industry intended to promote the collection and utilization of vascular device data across real-world data (RWD) sources. Completed RAPID projects included developing a minimum core data set to support consistent data collection, utilizing RWE to develop contemporary objective performance goals for peripheral interventions, and convening stakeholders to evaluate a safety signal.

This article summarizes RAPID’s recent work in elucidating the current landscape for global regulatory application of RWE. By gathering information on successful and unsuccessful industry experiences using RWE in different regulatory jurisdictions as well as the availability of region-specific RWE guidance,^1-26 our goal is to identify areas of opportunity for global RWE application, as well as gaps where additional outreach or education may be beneficial.

FDA REPORT: EXAMPLES OF RWE USED IN MEDICAL DEVICE REGULATORY DECISIONS

FDA released a report in March 2021 summarizing 90 device submissions supported by RWE.²⁷ The FDA report identifies submission number, sponsor, device, RWD source, premarket or postmarket use, and whether RWE was the primary clinical evidence source.

Regulatory submission types included premarket approval (PMA), de novo, 510(k), and Humanitarian Device Exemptions supported by a variety of RWD sources (eg, national/international registries, sponsor registries, claims data, and medical records). Premarket examples supported original device approval, indication expansion, and changes to labeling or instructions for use. Postmarket examples satisfied postapproval commitments and PMA conditions of approval. In some cases, these data sets were also used to meet non–United States regulatory goals, including continued marketing in Europe under the Medical Device Regulation (MDR) and supporting cardiovascular device approvals in Japan.

The RAPID Global Regulatory Acceptance working group reviewed the report in detail and identified 11 examples of international RWD used to support FDA regulatory decisions. During the May 2022, RAPID Think Tank meeting, the working group presented the 11 FDA international examples and one additional recent example (Table 1).²⁸ During a panel discussion, the RAPID Think Tank participants noted limitations in the visibility of examples in which RWE was accepted or not accepted by global regulators. The panel discussed success in using Japanese surveillance data (high quality with up to 8-year follow-up) and Chinese use of RWE for novel technologies in the Hainan region. Think Tank participants were polled to understand the range of RWE experiences. Most RWE experience has been from the United States, Europe, and Japan, and examples and information around the use of RWE within and among these regions would be helpful to industry (Figure 1 and Figure 2). Based on the polling results, the RAPID Global Regulatory Acceptance working group focused on reviewing global guidance documents and collecting additional examples of successful and unsuccessful use cases.

Figure 1. Results of poll regarding participants’ experience with successful RWE submissions across regions.

Figure 2. Results of poll to gauge options for improving industry’s understanding of submitting RWE.

AVAILABLE GUIDANCE IN USE OF RWE FOR REGULATORY APPROVAL

The working group also looked at available guidance documents for use of RWE. Although regulatory approval of medical devices has traditionally been obtained through prospective clinical trials, the rise of electronic capture of medical data through electronic health records and registries has increased the availability of RWD. As a result, it was recognized that a framework could be developed to allow for use of RWE for regulatory decision-making. Over the last decade, RWE guidance documents have been released by regulatory agencies from different regions including the United States, Japan, China, Canada, Australia, the European Union (EU), and the United Kingdom (see appendix).^1-26

Fortunately, these guidance documents share more similarities than differences. Many of these provide clear definitions of what constitutes RWD, when and how it can be used, and potential issues that could arise. Furthermore, the guidance documents define how RWD can be used to generate RWE for various purposes, including the use of RWE to support premarket submissions, postmarket surveillance studies, and pediatric marketing applications. Many of the documents have a strong focus on the quality of data used to generate RWE, noting that only high-quality data can be used for regulatory approval. Although that may seem obvious, there is no clear definition of high-quality data, and many documents go in-depth when discussing where shortfalls in RWD may exist, with some discussing methods to mitigate risk inherent in RWD.

In addition to guidance documents released by government agencies, the International Medical Device Regulators Forum (IMDRF) released a guide to assist manufacturers in using RWD. The IMDRF documents go into significant depth in using RWD, including assessing data quality from registry-based data, linkage of data sources, methodologies for use of registry data, finding signals in data, and much more. The IMDRF documents expound on frameworks that are present in other documents and can serve as an important adjunct source of information when using RWD.

Despite the many similarities, there are also some differences between guidance documents. For example, the Japanese and EU guidance documents emphasize registry data, the Canadian documents focus on pharmaceutical approval, and the EU and United Kingdom documents discuss using RWD clinical trials in greater depth. However, it is important to note that these differences should not impact the fundamentals of RWD usage for regulatory approval and can therefore be viewed as additional information based on the area of interest or approach being considered. Overall, the documents demonstrate agreement across regions on the ability to use RWD for regulatory approval.

RWE LANDSCAPE

The working group created a mind map of the RWE landscape (Figure 3) based on the findings in the global regulatory guidance documents, which were additionally supported by the RWE examples from FDA. The mind map served as a summary of the RWE landscape and was central to organizing key concepts. The usage of RWE in the regulatory domain supports product development/research, premarket decisions, and postmarket activities. In addition, RWE supports other business processes outside the regulatory domain such as economic analysis, reimbursement, and public health surveillance.

Figure 3. Mind map of the global RWE landscape. The diagram depicts the most commonly cited quality attributes in RWD that may be used as evidence in the various types of studies throughout the global RWE landscape. Note that not all of these were represented in each region that was assessed. Also depicted are the business processes supported by RWE, which include regulatory activities (eg, premarket authorization and postmarket surveillance and research) as well as nonregulatory activities (eg, product development/research, economic analysis, reimbursement, and public health surveillance).

The types of studies that utilized RWE were identified as feasibility planning, registry-embedded, external control of clinical trials, and pragmatic trials. The following characteristics were noted for RWD as the source of RWE: validity, completeness, data quality, and fit for purpose. Each of these characteristics are important to the relevance and reliability of RWD for use in regulatory decision-making as evidence to support premarket decisions as well as postmarket commitments.

RWE INDUSTRY SURVEYS

The RAPID Global Regulatory Acceptance working group conducted an industry survey to better understand RWE experiences and the uses of RWE for regulatory decision-making by gathering information on both positive and negative experiences with RWE to support regulatory decisions. A total of 37 RWE experiences were submitted between September 7, 2022, and October 14, 2022.

For the majority of submissions, the United States (n = 23) was the primary region in which RWE was used, along with Europe (n = 7) and Japan (n = 5), with single experiences reported in Canada and China. Aortic valve devices (n = 8) and coronary drug-eluting stents (n = 6) were the most common device types. Additional devices included angioplasty balloons, peripheral drug-coated balloons, peripheral bare-metal stents, peripheral drug-eluting stents, covered stents, peripheral atherectomy devices, mitral valve devices, and thrombectomy devices. RWE was used as the primary source of data in most cases (n = 30), with one additional submission utilizing RWE as both primary and supplemental data. Sources of RWD included society registries, electronic health records, manufacturer/sponsor registries, national registries, literature reviews, claims data, retrospective studies, health care databases, and physician-sponsored studies (Figure 4). Experiences included both premarket (n = 20) and postmarket (n = 17) submissions.

Figure 4. Survey results for RWD characteristics.

Of the 37 submissions, 31 RWE experiences were identified as successful. Estimated time savings using RWE compared to traditional independent industry studies varied, with eight respondents estimating 0 to 6 months, six estimating 6 to 12 months, seven estimating 12 to 24 months, and five estimating 2 to 5 years. Estimated cost savings ranged from $500K to $20M.

There were six RWE experiences that were identified as unsuccessful (5 United States submissions, 1 Japanese submission). Reported issues identified by regulators related to these unsuccessful submissions included access to data, data completeness, data quality, and inability to demonstrate conformance to Good Clinical Practice. However, despite these unsuccessful attempts, when asked what would encourage submitters to consider the use of RWE in subsequent regulatory submissions, many reaffirmed the importance of consultation mechanisms, development of clear regional guidance, increased transparency of data availability by RWD holders, and direction from regulators as to when the use of RWE should be considered.

Concerns reported in the survey that diminish enthusiasm for using RWE to support regulatory submissions include the perceived inability to use the same data across regions, lack of guidance from regulators, and concerns with data confidentiality. Moving forward, key considerations that may enhance the likelihood of successful submissions include formal and/or informal communication with regulators, ongoing discussions with regulatory bodies about RWE, and creation of a reusable infrastructure to answer multiple questions about real-world medical device safety and effectiveness.

Discussion

The review of global RWE guidance documents revealed no apparent substantive discrepancies or conflicts between regulatory bodies regarding the use of RWE for regulatory decision-making. However, many guidance documents provide general conceptual information about RWE without describing specific details or examples on the use or application of RWE within a certain jurisdiction. This higher-level focus is likely due to the relative novelty of RWE as a key component of the medical device regulatory landscape. Although many regulatory bodies support and encourage use of RWE, the mechanism of how specifically to incorporate RWE into a submission and what sources may qualify as high-quality RWD are provided broad latitude. This flexibility can be positive by allowing both regulators and industry to determine an appropriate use scenario that fits individual situations but also presents a challenge due to a lack of clarity on what may be acceptable for a given situation. Therefore, early discussion with regulators is especially important when considering the use of RWE to support a regulatory submission.

The 90 RWE cases shared by FDA provide examples of the conditions under which RWE was able to support a regulatory submission in the United States and can be used as a guide by industry. However, when considering use of RWE in other regions, there are limited examples available for industry to reference. Regulatory bodies and industry can help address this challenge by providing additional examples of the successful use of RWE to support regulatory decisions globally.

Another gap identified is the lack of examples around unsuccessful RWE submissions. These unsuccessful experiences can be valuable learning opportunities, often more so than successful examples, and can be used to identify challenges or limitations that still exist for RWE. By analyzing specific scenarios to identify and share common problem areas, industry and regulators can work together to identify pathways for overcoming remaining challenges, leading to more frequent and successful use of RWE in the future.

The potential benefits of RWE use were clearly identified through the industry survey, finding that RWE resulted in direct cost savings of $500K to $20M and time savings ranging from months to years. The primary risk of RWE usage in regulatory submissions is a lack of acceptance by the regulator. This risk may be mitigated by early and open communication between industry and regulators to identify concerns or questions about the proposed use of RWE and working collaboratively to identify the best path forward and minimize uncertainty.

CONCLUSION

Overall, the use of RWE to support regulatory decision-making is supported by many global regulatory authorities. Increased communication between industry and regulators and increased sharing of both successful and unsuccessful RWE experiences can continue to overcome challenges, improve the quality of RWD, and advance the global acceptance of RWE for regulatory decision-making.

1. US Food and Drug Administration. Use of real-world evidence to support regulatory decision-making for medical devices: guidance or industry and Food and Drug Administration staff. Accessed July 24, 2023 https://www.fda.gov/media/99447/download

2. US Food and Drug Administration. Submitting documents using real-world data and real-world evidence to FDA for drug and biological products: guidance for industry. September 2022. Accessed July 31, 2023. https://www.fda.gov/media/124795/download

3. US Food and Drug Administration. Data standards for drug and biological product submissions containing real-world data. October 2021. Accessed July 31, 2023. https://www.fda.gov/media/153341/download

4. US Food and Drug Administration. Real-world data: assessing electronic health records and medical claims data to support regulatory decision-making for drug and biological products. September 2021. Accessed July 31, 2023. https://www.fda.gov/media/152503/download

5. US Food and Drug Administration. Real world data: assessing registries to support regulatory decision making for drug and biological products: guidance for industry. November 2021. Accessed July 31, 2023. https://www.fda.gov/media/154449/download

6. US Food and Drug Administration. Considerations for the use of RWD and RWE to support regulatory decision-making for drug and biological products: guidance for industry. December 2021. Accessed July 31, 2023. https://www.fda.gov/media/154714/download

7. Pharmaceuticals and Medical Devices Agency. Points to consider for ensuring the reliability in utilization of registry data for applications. PSEHB/PED Notification No.0323-2. March 23, 2021. Accessed July 31, 2023. https://www.pmda.go.jp/files/000240811.pdf

8. Pharmaceuticals and Medical Devices Agency. Basic principles on utilization of registry for applications. PSEHB/PED Notification No.0323-1. March 23, 2021. Accessed July 31, 2023. https://www.pmda.go.jp/files/000240810.pdf

9. Pharmaceuticals and Medical Devices Agency. Guidelines for the conduct of pharmacoepidemiological studies in drug safety assessment with medical information databases. March 31, 2014. Accessed July 31, 2023. https://www.pmda.go.jp/files/000240951.pdf

10. International Medical Device Regulators Forum. Methodological principles in the use of international medical device registry data. March 16, 2017. Accessed July 31, 2023. https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-170316-methodological-principles.pdf 11

11. International Medical Device Regulators Forum. Tools for assessing the usability of registries in support of regulatory decision-making. March 27, 2018. Accessed July 31, 2023. https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-180327-usability-tools-n46.pdf

12. International Medical Device Regulators Forum. Principles of international system of registries linked to other data sources and tools. September 30, 2016. Accessed July 31, 2023. https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-160930-principles-system-registries.pdf

13. International Medical Device Regulators Forum. Clinical evaluation. October 10, 2019. Accessed July 31, 2023. https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-191010-mdce-n56.pdf

14. International Medical Device Regulators Forum. Clinical evidence—key definitions and concepts. October 10, 2019. Accessed July 31, 2023. https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-191010-mdce-n55.pdf

15. International Medical Device Regulators Forum. Post-market clinical follow-up studies. March 25, 2021. Accessed July 31, 2023. https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-210325-wng65.pdf

16. Government of Canada. Optimizing the use of real world evidence to inform regulatory decision-making. April 16, 2019. Accessed July 31, 2023. https://www.canada.ca/en/health-canada/services/drugs-health-products/drug-products/announcements/optimizing-real-world-evidence-regulatory-decisions.html

17. Government of Canada. Guidance on clinical evidence requirements for medical devices. November 15, 2022. Accessed July 31, 2023. https://www.canada.ca/en/health-canada/services/drugs-health-products/medical-devices/application-information/guidance-documents/clinical-evidence-requirements-medical-devices.html

18. Government of Canada. Elements of real world data/evidence quality throughout the prescription drug product life cycle. March 5, 2019. Accessed July 31, 2023. https://www.canada.ca/en/services/health/publications/drugs-health-products/real-world-data-evidence-drug-lifecycle-report.html

19. Government of Canada. Medical device real-world data and evidence in Canada: an environmental scan highlights. March 11, 2020. Accessed July 31, 2023. https://www.canada.ca/en/health-canada/corporate/transparency/regulatory-transparency-and-openness/improving-review-drugs-devices/real-world-evidence-medical-device-strategy/environmental-scan.html

20. European Medicines Agency. Guideline on registry-based studies. September 24, 2020. Accessed July 31, 2023. https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-registry-based-studies_en.pdf

21. European Medicines Agency. Post-authorisation. Accessed July 31, 2023. https://www.ema.europa.eu/en/human-regulatory/post-authorisation#_blank

22. Australia Government; Department of Health and Aged Care. Joint TGA-Medicines Australia guidelines for the design and conduct of company-sponsored post-marketing surveillance. August 24. 2021. Accessed July 31, 2023. https://www.tga.gov.au/resources/publication/publications/joint-tga-medicines-australia-guidelines-design-and-conduct-company-sponsored-post-marketing-surveillance-studies

23. Australia Government; Department of Health and Aged Care. Risk management plans for medicines and biologicals. March 29, 2019. Accessed July 31, 2023. https://www.tga.gov.au/resources/resource/guidance/risk-management-plans-medicines-and-biologicals

24. Australian Government; Department of Health. Therapeutic Goods Administration: International Engagement Strategy 2021-2025: advancing Australia’s health through international regulatory engagement. Accessed July 31, 2023. https://www.tga.gov.au/sites/default/files/international-engagement-strategy-2021-2025.pdf

25. Medicines and Healthcare Products Regulatory Agency. MHRA draft guidance on randomised controlled trials generating real-world evidence to support regulatory decisions. October 30, 2020. Updated December 16, 2021. Accessed July 31, 2023. https://www.gov.uk/government/consultations/mhra-draft-guidance-on-randomised-controlled-trials-generating-real-world-evidence-to-support-regulatory-decisions

26. National Institute for Health and Care Excellence. Evidence standards framework (ESF) for digital health technologies. Accessed July 31, 2023. https://www.nice.org.uk/about/what-we-do/our-programmes/evidence-standards-framework-for-digital-health-technologies

27. US Food and Drug Administration. Examples of real-world evidence (RWE) used in medical device regulatory decisions. Accessed July 24, 2023. https://www.fda.gov/media/146258/download

28. MDEpiNet. Registry-supported prospective trials virtual think tank III: working group updates on 2022 deliverables & beyond. Presented May 5, 2022 virtually. Accessed July 24, 2023. https://www.mdepinet.net/event-details/registry-supported-prospective-trials-think-tank-iv-work-group-deliverables-2022-plans-2023