Impact of searching clinical trial registries in systematic reviews of pharmaceutical treatments: methodological systematic review and reanalysis of meta-analyses

Objective To evaluate the impact of searching clinical trial registries in systematic reviews. Design Methodological systematic review and reanalyses of meta-analyses. Data sources Medline was searched to identify systematic reviews of randomised controlled trials (RCTs) assessing pharmaceutical treatments published between June 2014 and January 2015. For all systematic reviews that did not report a trial registry search but reported the information to perform it, the World Health Organization International Trials Registry Platform (WHO ICTRP search portal) was searched for completed or terminated RCTs not originally included in the systematic review. Data extraction For each systematic review, two researchers independently extracted the outcomes analysed, the number of patients included, and the treatment effect estimated. For each RCT identified, two researchers independently determined whether the results were available (ie, posted, published, or available on the sponsor website) and extracted the data. When additional data were retrieved, we reanalysed meta-analyses and calculated the weight of the additional RCTs and the change in summary statistics by comparison with the original meta-analysis. Results Among 223 selected systematic reviews, 116 (52%) did not report a search of trial registries; 21 of these did not report the information to perform the search (key words, search date). A search was performed for 95 systematic reviews; for 54 (57%), no additional RCTs were found and for 41 (43%) 122 additional RCTs were identified. The search allowed for increasing the number of patients by more than 10% in 19 systematic reviews, 20% in 10, 30% in seven, and 50% in four. Moreover, 63 RCTs had results available; the results for 45 could be included in a meta-analysis. 14 systematic reviews including 45 RCTs were reanalysed. The weight of the additional RCTs in the recalculated meta-analyses ranged from 0% to 58% and was greater than 10% in five of 14 systematic reviews, 20% in three, and 50% in one. The change in summary statistics ranged from 0% to 29% and was greater than 10% for five of 14 systematic reviews and greater than 20% for two. However, none of the changes to summary effect estimates led to a qualitative change in the interpretation of the results once the new trials were added. Conclusions Trial registries are an important source for identifying additional RCTs. The additional number of RCTs and patients included if a search were performed varied across systematic reviews.

[1]  S. Ebrahim,et al.  Meta-analyses with industry involvement are massively published and report no caveats for antidepressants. , 2016, Journal of clinical epidemiology.

[2]  T. Groves.  Mandatory disclosure of trial results for drugs and devices , 2008, BMJ : British Medical Journal.

[3]  D. Cook,et al.  Systematic Reviews: Synthesis of Best Evidence for Clinical Decisions , 1997, Annals of Internal Medicine.

[4]  T. Jefferson,et al.  Neuraminidase inhibitors for preventing and treating influenza in healthy adults. , 2006, The Cochrane database of systematic reviews.

[5]  Su Golder,et al.  Reporting of Adverse Events in Published and Unpublished Studies of Health Care Interventions: A Systematic Review , 2016, PLoS medicine.

[6]  L. Hooft,et al.  Identification of Additional Trials in Prospective Trial Registers for Cochrane Systematic Reviews , 2012, PloS one.

[7]  P. Glasziou,et al.  Avoidable waste in the production and reporting of research evidence. , 2009, Obstetrics and gynecology.

[8]  A. Stewart.  Mandating HPV vaccination--private rights, public good. , 2007, The New England journal of medicine.

[9]  C. Gamble,et al.  Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review , 2013, PloS one.

[10]  David Moher,et al.  Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study , 2016, PLoS medicine.

[11]  Isabelle Boutron,et al.  Timing and Completeness of Trial Results Posted at ClinicalTrials.gov and Published in Journals , 2013, PLoS medicine.

[12]  Steven Duffy,et al.  Searching ClinicalTrials.gov and the International Clinical Trials Registry Platform to inform systematic reviews: what are the optimal search approaches? , 2014, Journal of the Medical Library Association : JMLA.

[13]  P. Easterbrook,et al.  Publication bias in clinical research , 1991, The Lancet.

[14]  J. Ioannidis,et al.  The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration , 2009, Annals of Internal Medicine [serial online].

[15]  Isabelle Boutron,et al.  Single-Center Trials Show Larger Treatment Effects Than Multicenter Trials: Evidence From a Meta-epidemiologic Study , 2011, Annals of Internal Medicine.

[16]  Ana Marusic,et al.  Clinical trial registration--looking back and moving ahead. , 2007, The New England journal of medicine.

[17]  C. Bias.  The Cochrane Collaboration's tool for assessing risk of bias in randomised trials , 2011 .

[18]  John Hoey,et al.  Clinical trial registration: a statement from the International Committee of Medical Journal Editors. , 2005, The New England journal of medicine.

[19]  H. Bastian,et al.  Seventy-Five Trials and Eleven Systematic Reviews a Day: How Will We Ever Keep Up? , 2010, PLoS medicine.

[20]  Lisa Bero,et al.  Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses , 2012, BMJ : British Medical Journal.

[21]  Christopher W. Jones,et al.  Systematic Reviews Published in Emergency Medicine Journals Do Not Routinely Search Clinical Trials Registries: A Cross-Sectional Analysis. , 2015, Annals of emergency medicine.

[22]  Bartha M. Knoppers,et al.  Data sharing, year 1--access to data from industry-sponsored clinical trials. , 2014, The New England journal of medicine.

[23]  F. Song,et al.  Dissemination and publication of research findings: an updated review of related biases. , 2010, Health technology assessment.

[24]  B. Wieseler,et al.  Impact of Inclusion of Industry Trial Results Registries as an Information Source for Systematic Reviews , 2014, PloS one.

[25]  E. Cobo,et al.  A hazard ratio was estimated by a ratio of median survival times, but with considerable uncertainty. , 2014, Journal of clinical epidemiology.

[26]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. , 2009, Journal of clinical epidemiology.

[27]  Tom Jefferson,et al.  Neuraminidase inhibitors for preventing and treating influenza in healthy adults: systematic review and meta-analysis , 2009, BMJ : British Medical Journal.

[28]  David Moher,et al.  Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews , 2007, BMC medical research methodology.

[29]  J. Sterne,et al.  The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials , 2011, BMJ : British Medical Journal.

[30]  J. Higgins,et al.  Cochrane Handbook for Systematic Reviews of Interventions , 2010, International Coaching Psychology Review.

[31]  Lisa Bero,et al.  Reporting Bias in Drug Trials Submitted to the Food and Drug Administration: Review of Publication and Presentation , 2008, PLoS medicine.

[32]  Christopher W. Jones,et al.  Clinical trials registries are under-utilized in the conduct of systematic reviews: a cross-sectional analysis , 2014, Systematic Reviews.

[33]  Harlan M Krumholz,et al.  Increasing value and reducing waste: addressing inaccessible research , 2014, The Lancet.

[34]  David Moher,et al.  Epidemiology and Reporting Characteristics of Systematic Reviews , 2007, PLoS medicine.

[35]  Sally Hopewell,et al.  Publication bias in clinical trials due to statistical significance or direction of trial results. , 2009, The Cochrane database of systematic reviews.

[36]  D. Moher,et al.  Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement , 2009, BMJ : British Medical Journal.

[37]  Tony Tse,et al.  Trial Reporting in ClinicalTrials.gov - The Final Rule. , 2016, The New England journal of medicine.

[38]  I. Boutron,et al.  Impact of sending email reminders of the legal requirement for posting results on ClinicalTrials.gov: cohort embedded pragmatic randomized controlled trial , 2014, BMJ : British Medical Journal.

[39]  G. Antes,et al.  Extent of Non-Publication in Cohorts of Studies Approved by Research Ethics Committees or Included in Trial Registries , 2014, PloS one.