Considerations in characterizing real‐world data relevance and quality for regulatory purposes: A commentary

The 21st Century Cures Act of 2016 provided a framework to the US Food and Drug Administration (FDA) to rapidly move treatments to patients.1 The increased acceptability of real‐world data (RWD) sources allows for innovative ways to study products and has the potential to reduce trial costs. Published papers provide guidance regarding data quality issues, reproducibility, and validity assessment.2 Rapid evolvement of electronic health records (EHRs) encourages greater consideration of their use in research.1, 2, 3, 4, 5, 6 For years, the FDA has relied on epidemiological studies of postapproval product safety using RWD5, 6 (eg, administrative claims and EHR) and for device effectiveness studies4; however, regulatory use for evaluating drug effectiveness has been rare. As part of the Prescription Drug User Fee Act (PDUFA VI),3 use of RWD is being considered for potential contributions to evaluating effectiveness and safety of new indications for approved products and to satisfy postapproval study requirements. Recently, the Duke Margolis Center for Health Policy held workshops and issued two paper on this topic.5, 6 The first paper focused on defining RWD as data routinely collected pertinent to patient health status and/or delivery of care, and the use of RWD in regulatory and clinical contexts.5 The second white paper from the October 1, 2018, workshop focused on data relevancy and quality, including cleaning, transforming, and linking RWD to characterize RWD sources as “fit for regulatory purpose.”6 These papers offer a practical “commonsense” high‐level view of primary data and methods considerations for RWD use from a regulatory perspective, facilitating discussion around regulatory uses of RWD within the research community and industry. However, salient points are missing from the papers and the RWD discussions among FDA, researchers, and industry. Here, we provide a commentary on the data considerations discussed in the white papers and highlight pertinent considerations with respect to RWD in the context of whether data are relevant, representative, and robust. 1.1. Data relevance The recent white paper defines data relevance dimensions including representativeness of the population of interest, critical data field availability, accurate linking at the patient level with multiple data sources, and adequate sample size and follow‐up time to demonstrate expected treatment effects.6 Guidance from FDA on how to ensure RWD are fit for purpose and adequate to support regulatory decisions would be helpful on each dimension. Determining if RWD is fit for regulatory purpose is a “contextual exercise” where the specific research question, regulatory use, and data characteristics drive what meaningful conclusions can be drawn.6 Covariates may be critical for one research question but not another. Exposures and outcomes should be well defined when part of the research question but may not be critical for natural history studies. There is no “one‐size‐fits‐all” approach, and critical data components should be evaluated for each research question and regulatory use.7 A framework is needed to guide choice and evaluation of critical data elements for specific research questions for regulatory use. Representativeness of the population of interest is gauged in many ways. Recent FDA guidance on Patient Focused Drug Development suggests a statistical sampling approach be used to obtain patient experience data representative of the target population.8 However, most US real‐world databases use administrative claims or EHR for patients seeking medical attention. These RWD sources should be considered broadly representative of the population eligible for using most, if not all, new products and services. “Representativeness” should be assessed broadly in the context of likely product users with some diversity in geography, health status, and health care system as appropriate for the specific research question and regulatory context. While data linkage is likely to limit the eligible sample, it may be needed to increase the informative nature of RWD, especially with increasing evaluations to support precision medicine. Sample size should be derived based on anticipated treatment effects for studies of treatment effectiveness or safety, whether comparative or not, to ensure appropriate precision of estimates. For rare diseases, there should be flexibility given data sparseness worldwide, as indicated in the FDA guidance on rare disease.8 Additional guidance would be useful regarding how “accurate linking” should be assessed since linking 100% of patients with administrative claims and EHR is impractical. Would FDA accept limited linked data if it was supplemental to cruder variables in the full dataset? Would a subset of 60% be adequate? In the context of probabilistic linkage, what level of certainty would constitute adequate linkage? Salience of linkable individuals to the specific research question should be considered in this determination and pre‐specified sensitivity analyses should help assess robustness of results and conclusions.9, 10