Larger effect sizes in nonrandomized studies are associated with higher rates of EMA licensing approval.

OBJECTIVES The aim of this study was to evaluate how often the European Medicines Agency (EMA) has authorized drugs based on nonrandomized studies and whether there is an association between treatment effects and EMA preference for further testing in randomized clinical trials (RCTs). STUDY DESIGN AND SETTING We reviewed all initial marketing authorizations in the EMA database on human medicines between 1995 and 2015 and included authorizations granted without randomized data. We extracted data on treatment effects and EMA preference for further testing in RCTs. RESULTS Of 723 drugs, 51 were authorized based on nonrandomized data. These 51 drugs were licensed for 71 indications. In the 51 drug-indication pairs with no preference for further RCT testing, effect estimates were large [odds ratio (OR): 12.0 (95% confidence interval {CI}: 8.1-17.9)] compared to effect estimates in the 20 drug-indication pairs for which future RCTs were preferred [OR: 4.3 (95% CI 2.8-6.6)], with a significant difference between effects (P = 0.0005). CONCLUSION Nonrandomized data were used for 7% of EMA drug approvals. Larger effect sizes were associated with greater likelihood of approval based on nonrandomized data alone. We did not find a clear treatment effect threshold for drug approval without RCT evidence.

[1]  J. Ioannidis,et al.  The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration , 2009, Annals of Internal Medicine [serial online].

[2]  Mark C. Wilson Evidence-Based Medicine: How to Practice and Teach EBM , 2001, ACP Journal Club.

[3]  B. Djulbegovic,et al.  Benchmarks for detecting ‘breakthroughs’ in clinical trials: empirical assessment of the probability of large treatment effects using kernel density estimation , 2014, BMJ Open.

[4]  Gianluca Baio,et al.  Regulatory approval of pharmaceuticals without a randomised controlled study: analysis of EMA and FDA approvals 1999–2014 , 2016, BMJ Open.

[5]  Janet Woodcock,et al.  Expediting drug development--the FDA's new "breakthrough therapy" designation. , 2013, The New England journal of medicine.

[6]  I. Chalmers Invalid health information is potentially lethal , 2001, BMJ : British Medical Journal.

[7]  B. Djulbegovic,et al.  Evidence-based medicine for rare diseases: implications for data interpretation and clinical trial design. , 2007, Cancer control : journal of the Moffitt Cancer Center.

[8]  Iztok Hozo,et al.  When Should Potentially False Research Findings Be Considered Acceptable? , 2007, PLoS medicine.

[9]  B. Djulbegovic,et al.  Comparator bias: why comparisons must address genuine uncertainties , 2013, Journal of the Royal Society of Medicine.

[10]  N. Black,et al.  The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. , 1998, Journal of epidemiology and community health.

[11]  J. Ioannidis,et al.  Timing and Characteristics of Cumulative Evidence Available on Novel Therapeutic Agents Receiving Food and Drug Administration Accelerated Approval , 2017, The Milbank quarterly.

[12]  Gordon H Guyatt,et al.  GRADE guidelines: 9. Rating up the quality of evidence. , 2011, Journal of clinical epidemiology.

[13]  K. Kern Trial Design and Efficacy Thresholds for Granting Breakthrough Therapy Designation in Oncology. , 2016, Journal of oncology practice.

[14]  R. Lanzara Weber's law modeled by the mathematical description of a beam balance. , 1994, Mathematical biosciences.

[15]  I. Chalmers,et al.  Patients and the public deserve big changes in evaluation of drugs , 2009, BMJ : British Medical Journal.

[16]  Casting and drawing lots: a time honoured way of dealing with uncertainty and ensuring fairness , 2001, BMJ : British Medical Journal.

[17]  Gnosis and random allotment. , 1981, Controlled clinical trials.

[18]  P. Bossuyt,et al.  Grading quality of evidence and strength of recommendations in clinical practice guidelines Part 3 of 3. The GRADE approach to developing recommendations , 2011, Allergy.

[19]  S. S. Stevens Neural events and the psychophysical law. , 1970, Science.

[20]  Nicky J Welton,et al.  Evaluating novel agent effects in multiple‐treatments meta‐regression , 2010, Statistics in medicine.

[21]  R. Banzi,et al.  Conditional approval of medicines by the EMA , 2017, British Medical Journal.

[22]  Sharon E Straus,et al.  Evidence-Based Medicine: How to Practice and Teach It , 2010 .

[23]  J. Ioannidis,et al.  Very large treatment effects in randomised trials as an empirical marker to indicate whether subsequent trials are necessary: meta-epidemiological assessment , 2016, British Medical Journal.

[24]  Paul Glasziou,et al.  When are randomised trials unnecessary? Picking signal from noise , 2007, BMJ : British Medical Journal.

[25]  M. Hernán,et al.  ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions , 2016, British Medical Journal.

[26]  Francisco Cervantes,et al.  Imatinib compared with interferon and low-dose cytarabine for newly diagnosed chronic-phase chronic myeloid leukemia. , 2003, The New England journal of medicine.

[27]  Gordon H Guyatt,et al.  GRADE guidelines: 5. Rating the quality of evidence--publication bias. , 2011, Journal of clinical epidemiology.

[28]  John P A Ioannidis,et al.  Empirical evaluation of very large treatment effects of medical interventions. , 2012, JAMA.

[29]  Benjamin Djulbegovic,et al.  Articulating and responding to uncertainties in clinical research. , 2007, The Journal of medicine and philosophy.