The significant cost of systematic reviews and meta-analyses: A call for greater involvement of machine learning to assess the promise of clinical trials

Background More than 90% of clinical-trial compounds fail to demonstrate sufficient efficacy and safety. To help alleviate this issue, systematic literature review and meta-analysis (SLR), which synthesize current evidence for a research question, can be applied to preclinical evidence to identify the most promising therapeutics. However, these methods remain time-consuming and labor-intensive. Here, we introduce an economic formula to estimate the expense of SLR for academic institutions and pharmaceutical companies. Methods We estimate the manual effort involved in SLR by quantifying the amount of labor required and the total associated labor cost. We begin with an empirical estimation and derive a formula that quantifies and describes the cost. Results The formula estimated that each SLR costs approximately $141,194.80. We found that on average, the ten largest pharmaceutical companies publish 118.71 and the ten major academic institutions publish 132.16 SLRs per year. On average, the total cost of all SLRs per year to each academic institution amounts to $18,660,304.77 and for each pharmaceutical company is $16,761,234.71. Discussion It appears that SLR is an important, but costly mechanisms to assess the totality of evidence. Conclusions With the increase in the number of publications, the significant time and cost of SLR may pose a barrier to their consistent application to assess the promise of clinical trials thoroughly. We call on investigators and developers to develop automated solutions to help with the assessment of preclinical evidence particularly. The formula we introduce provides a cost baseline against which the efficiency of automation can be measured.

[1]  Carole Federico,et al.  Consider drug efficacy before first-in-human trials , 2017, Nature.

[2]  Brian Chen,et al.  "Right-to-Try" Legislation: Progress or Peril? , 2015, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[3]  Gillian L. Currie,et al.  Systematic Review and Meta-analysis: Important Tools in Understanding Drug Development for Stroke , 2017 .

[4]  G. Norman Drugs, Devices, and the FDA: Part 1: An Overview of Approval Processes for Drugs. , 2016 .

[5]  S. Gopalakrishnan,et al.  Systematic Reviews and Meta-analysis: Understanding the Best Evidence in Primary Healthcare , 2013, Journal of family medicine and primary care.

[6]  Andrew S.C. Rice,et al.  Animal models and the prediction of efficacy in clinical trials of analgesic drugs: A critical appraisal and call for uniform reporting standards , 2008, PAIN.

[7]  N. Laird,et al.  Meta-analysis in clinical trials revisited. , 2015, Contemporary clinical trials.

[8]  A B Haidich,et al.  Meta-analysis in medical research. , 2010, Hippokratia.

[9]  Andrew W. Brown,et al.  Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry , 2017, BMJ Open.

[10]  Jessica Gurevitch,et al.  Meta-analysis and the science of research synthesis , 2018, Nature.

[11]  Matthew Michelson,et al.  A Deep Learning Method to Automatically Identify Reports of Scientifically Rigorous Clinical Research from the Biomedical Literature: Comparative Analytic Study , 2018, Journal of medical Internet research.

[12]  Van Norman Ga Drugs, Devices, and the FDA: Part 1: An Overview of Approval Processes for Drugs. , 2016 .

[13]  Jop Vrieze Meta-analyses were supposed to end scientific debates. Often, they only cause more controversy , 2018, Science.

[14]  Matthew Michelson Automating Meta-Analyses of Randomized Clinical Trials: A First Look , 2014, AAAI Fall Symposia.

[15]  Suhail A R Doi,et al.  Meta-analysis of heterogeneous clinical trials: an empirical example. , 2011, Contemporary clinical trials.

[16]  Steven Minton,et al.  AI2 LEVERAGING MACHINE-ASSISTANCE TO REPLICATE A SYSTEMATIC REVIEW , 2019, Value in Health.

[17]  R. Day,et al.  No evidence or no alternative? Taking responsibility for off‐label prescribing , 2012, Internal medicine journal.

[18]  Ethan M Balk,et al.  A Web-based archive of systematic review data , 2012, Systematic Reviews.

[19]  W. Bretz,et al.  Red Marine Algae Lithothamnion calcareum Supports Dental Enamel Mineralization , 2023, Marine drugs.

[20]  G. Norman,et al.  Drugs, Devices, and the FDA: Part 2: An Overview of Approval Processes: FDA Approval of Medical Devices , 2016 .

[21]  D. Altshuler,et al.  Validating therapeutic targets through human genetics , 2013, Nature Reviews Drug Discovery.

[22]  Seema Rawat,et al.  Publish or perish: Where are we heading? , 2014, Journal of research in medical sciences : the official journal of Isfahan University of Medical Sciences.

[23]  Laura A. Levit,et al.  Finding what works in health care : standards for systematic reviews , 2011 .

[24]  Ewoud Schuit,et al.  Network meta-analyses performed by contracting companies and commissioned by industry , 2016, Systematic Reviews.

[25]  Malcolm Macleod,et al.  How can we improve the pre-clinical development of drugs for stroke? , 2007, Trends in Neurosciences.

[26]  Evans Whitaker UCSF Guides: Systematic Review: When will I be finished? , 2015 .

[27]  D. Moher,et al.  CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials , 2010, Journal of pharmacology & pharmacotherapeutics.

[28]  M. Ghert,et al.  Lost in translation: animal models and clinical trials in cancer treatment. , 2014, American journal of translational research.