Evaluating the Use of Nonrandomized Real‐World Data Analyses for Regulatory Decision Making

The analysis of longitudinal healthcare data outside of highly controlled parallel‐group randomized trials, termed real‐world evidence (RWE), has received increasing attention in the medical literature. In this paper, we discuss the potential role of RWE in drug regulation with a focus on the analysis of healthcare databases. We present several cases in which RWE is already used and cases in which RWE could potentially support regulatory decision making. We summarize key issues that investigators and regulators should consider when designing or evaluating such studies, and we propose a structured process for implementing analyses that facilitates regulatory review. We evaluate the empirical evidence base supporting the validity, transparency, and reproducibility of RWE from analysis of healthcare databases and discuss the work that still needs to be done to ensure that such analyses can provide decision‐ready evidence on the effectiveness and safety of treatments.

[1]  L. Howie,et al.  A 25-Year Experience of US Food and Drug Administration Accelerated Approval of Malignant Hematology and Oncology Drugs and Biologics: A Review , 2018, JAMA oncology.

[2]  Marsha E Reichman,et al.  Dabigatran and postmarketing reports of bleeding. , 2013, The New England journal of medicine.

[3]  James R. Rogers,et al.  Cardiovascular Safety of Tocilizumab Versus Tumor Necrosis Factor Inhibitors in Patients With Rheumatoid Arthritis: A Multi‐Database Cohort Study , 2017, Arthritis & rheumatology.

[4]  James M. Robins,et al.  Observational Studies Analyzed Like Randomized Experiments: An Application to Postmenopausal Hormone Therapy and Coronary Heart Disease , 2008, Epidemiology.

[5]  R Platt,et al.  Real World Data in Adaptive Biomedical Innovation: A Framework for Generating Evidence Fit for Decision‐Making , 2016, Clinical pharmacology and therapeutics.

[6]  Samy Suissa,et al.  Immortal time bias in observational studies of drug effects , 2007, Pharmacoepidemiology and drug safety.

[7]  S. Schneeweiss,et al.  Claims‐based studies of oral glucose‐lowering medications can achieve balance in critical clinical variables only observed in electronic health records , 2018, Diabetes, obesity & metabolism.

[8]  A. Avogaro,et al.  Analyses of Results From Cardiovascular Safety Trials With DPP-4 Inhibitors: Cardiovascular Outcomes, Predefined Safety Outcomes, and Pooled Analysis and Meta-analysis , 2016, Diabetes Care.

[9]  R. Guthrie Canagliflozin and cardiovascular and renal events in type 2 diabetes , 2018, Postgraduate medicine.

[10]  John P. A. Ioannidis,et al.  What does research reproducibility mean? , 2016, Science Translational Medicine.

[11]  D. Karnad,et al.  The Genesis of Cardiovascular Safety Regulatory Landscapes for New Antidiabetic Drugs for Type 2 Diabetes , 2017 .

[12]  Jeremy A. Rassen,et al.  Transparency and reproducibility of published observational cohort studies , 2015 .

[13]  S. Pocock,et al.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. , 2007, Preventive medicine.

[14]  S. Yusuf,et al.  Dual inhibition of the renin–angiotensin system in high-risk diabetes and risk for stroke and other outcomes: results of the ONTARGET trial , 2013, Journal of hypertension.

[15]  Sebastian Schneeweiss,et al.  When and How Can Real World Data Analyses Substitute for Randomized Controlled Trials? , 2017, Clinical pharmacology and therapeutics.

[16]  R. Simon,et al.  The role of nonrandomized trials in the evaluation of oncology drugs , 2015, Clinical pharmacology and therapeutics.

[17]  Danielle E. Clarkesmith,et al.  Cochrane Database of Systematic Reviews Educational and behavioural interventions for anticoagulant therapy in patients with atrial fibrillation (Review) , 2017 .

[18]  Robert C. Wolpert,et al.  A Review of the , 1985 .

[19]  Sebastian Schneeweiss,et al.  Use of Health Care Databases to Support Supplemental Indications of Approved Medications , 2018, JAMA internal medicine.

[20]  Suzanne Bonamici Text - H.R.34 - 114th Congress (2015-2016): 21st Century Cures Act , 2016 .

[21]  S. Cole,et al.  Overadjustment Bias and Unnecessary Adjustment in Epidemiologic Studies , 2009, Epidemiology.

[22]  D. Solomon,et al.  Assessment of Cardiovascular Risk in Older Patients With Gout Initiating Febuxostat Versus Allopurinol: Population-Based Cohort Study , 2018, Circulation.

[23]  J. Gagne,et al.  Safety assessment of niacin in the US Food and Drug Administration's mini‐sentinel system , 2018, Pharmacoepidemiology and drug safety.

[24]  F. J. Sasinowski,et al.  Quantum of Effectiveness Evidence in FDA’s Approval of Orphan Drugs , 2012 .

[25]  Samy Suissa,et al.  Immortal time bias in pharmaco-epidemiology. , 2008, American journal of epidemiology.

[26]  J. Avorn,et al.  Paradoxical Relations of Drug Treatment with Mortality in Older Persons , 2001, Epidemiology.

[27]  J M Rowe,et al.  Blinatumomab vs historical standard therapy of adult relapsed/refractory acute lymphoblastic leukemia , 2016, Blood Cancer Journal.

[28]  S. Pocock,et al.  Guidelines for Reporting Observational Studies , 2007 .

[29]  Chittaranjan Andrade,et al.  Confounding , 2007, Indian journal of psychiatry.

[30]  John D Seeger,et al.  Aprotinin during coronary-artery bypass grafting and risk of death. , 2008, The New England journal of medicine.

[31]  S. Yusuf,et al.  Dabigatran versus warfarin in patients with atrial fibrillation. , 2009, The New England journal of medicine.

[32]  R. Martineau,et al.  A comparison of aprotinin and lysine analogues in high-risk cardiac surgery. , 2008, The New England journal of medicine.

[33]  J. Rassen,et al.  Prospective Cohort Studies of Newly Marketed Medications: Using Covariate Data to Inform the Design of Large-Scale Studies , 2014, Epidemiology.

[34]  S. Schneeweiss Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics , 2006, Pharmacoepidemiology and drug safety.

[35]  K. Huybrechts,et al.  Safety and effectiveness of dabigatran and warfarin in routine care of patients with atrial fibrillation , 2015, Thrombosis and Haemostasis.

[36]  Guidance for Industry E 10 Choice of Control Group and Related Issues in Clinical Trials , 2001 .

[37]  Kenneth G Saag,et al.  Cardiovascular Safety of Febuxostat or Allopurinol in Patients with Gout , 2018, The New England journal of medicine.

[38]  T C Chalmers,et al.  Randomized versus historical controls for clinical trials. , 1982, The American journal of medicine.

[39]  Lisa M. LaVange,et al.  Multidimensional Evidence Generation and FDA Regulatory Decision Making: Defining and Using "Real-World" Data. , 2017, JAMA.

[40]  George Davey Smith,et al.  Negative control exposures in epidemiologic studies. , 2012, Epidemiology.

[41]  R. Califf,et al.  Real-World Evidence - What Is It and What Can It Tell Us? , 2016, The New England journal of medicine.

[42]  JA Rassen,et al.  Transparency and Reproducibility of Observational Cohort Studies Using Large Healthcare Databases. , 2016, Clinical pharmacology and therapeutics.

[43]  H. Eichler,et al.  Adaptive Licensing: Taking the Next Step in the Evolution of Drug Approval , 2012, Clinical pharmacology and therapeutics.

[44]  J. Ioannidis Why Most Published Research Findings Are False , 2005, PLoS medicine.

[45]  W Goettsch,et al.  From adaptive licensing to adaptive pathways: Delivering a flexible life‐span approach to bring new drugs to patients , 2015, Clinical pharmacology and therapeutics.

[46]  Jeremy Sugarman,et al.  Ethics and regulatory complexities for pragmatic clinical trials. , 2014, JAMA.

[47]  Duke-Margolis Characterizing RWD Quality and Relevancy for Regulatory Purposes , 2018 .

[48]  L. Bero,et al.  Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. , 2014, The Cochrane database of systematic reviews.

[49]  W Katherine Yih,et al.  Intussusception risk after rotavirus vaccination in U.S. infants. , 2014, The New England journal of medicine.

[50]  Laura McDonald,et al.  Real-world data in the United Kingdom: opportunities and challenges , 2016, BMC Medicine.

[51]  D. Karnad,et al.  Cardiovascular Safety in Drug Development and Therapeutic Use , 2017, Springer International Publishing.

[52]  C Michael Stein,et al.  COX-2 selective non-steroidal anti-inflammatory drugs and risk of serious coronary heart disease , 2002, The Lancet.

[53]  Douglas G Altman,et al.  [The Strengthening the Reporting of Observational Studies in Epidemiology [STROBE] statement: guidelines for reporting observational studies]. , 2007, Gaceta sanitaria.

[54]  W. Klapper,et al.  Blinatumomab versus Chemotherapy for Advanced Acute Lymphoblastic Leukemia , 2017, The New England journal of medicine.

[55]  J. Avorn,et al.  A review of uses of health care utilization databases for epidemiologic research on therapeutics. , 2005, Journal of clinical epidemiology.

[56]  A. Alavi,et al.  Opportunities and Challenges , 1998, In Vitro Diagnostic Industry in China.

[57]  R. J. Hayes,et al.  Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. , 1995, JAMA.

[58]  F. Dominici,et al.  Reproducible epidemiologic research. , 2006, American journal of epidemiology.

[59]  S S Ellenberg,et al.  Placebo-Controlled Trials and Active-Control Trials in the Evaluation of New Treatments. Part 1: Ethical and Scientific Issues , 2000, Annals of Internal Medicine.

[60]  S. Schneeweiss,et al.  Cardiovascular outcomes associated with canagliflozin versus other non-gliflozin antidiabetic drugs: population based cohort study , 2018, British Medical Journal.

[61]  Sebastian Schneeweiss,et al.  Selective prescribing led to overestimation of the benefits of lipid-lowering drugs. , 2006, Journal of clinical epidemiology.

[62]  M. Lipsitch,et al.  Negative Controls: A Tool for Detecting Confounding and Bias in Observational Studies , 2010, Epidemiology.