Machine Learning in Evidence Synthesis Research

In this chapter, we will explore how Systemic Reviews (SR) are traditionally conducted and how the process of arriving at a valuable SR can be made more efficient and less error prone using Machine Learning (ML) techniques. As the integration of ML at the screening stage of SRs has reached the highest level of maturity, we will explain the techniques utilized. We will further describe the extraction process from primary studies supported by ML techniques. The discussion of pitfalls when conducting SRs concludes the chapter, specifically how ML can address bias. Lastly, we address the inherent limitations of artificial intelligence in healthcare with a special emphasis on ML for the use in SRs.

[1]  T. Greenhalgh,et al.  Realist review - a new method of systematic review designed for complex policy interventions , 2005, Journal of health services research & policy.

[2]  P. Tugwell,et al.  Including non‐randomized studies on intervention effects , 2019, Cochrane Handbook for Systematic Reviews of Interventions.

[3]  F. Hua,et al.  Reporting quality of randomized controlled trial abstracts: survey of leading general dental journals. , 2015, Journal of the American Dental Association.

[4]  R. Weyant,et al.  Nonrestorative Treatments for Caries: Systematic Review and Network Meta-analysis , 2018, Journal of dental research.

[5]  Brian E. Howard,et al.  SWIFT-Review: a text-mining workbench for systematic review , 2016, Systematic Reviews.

[6]  David L. Sackett,et al.  Evidence based medicine: What it is and what it isn't (reprinted from BMJ, vol 312, pg 71-72, 1996) , 2007 .

[7]  Carla E. Brodley,et al.  Who Should Label What? Instance Allocation in Multiple Expert Active Learning , 2011, SDM.

[8]  Carla E. Brodley,et al.  Modeling annotation time to reduce workload in comparative effectiveness reviews , 2010, IHI.

[9]  D. Moher,et al.  A scoping review of rapid review methods , 2015, BMC Medicine.

[10]  William R. Hersh,et al.  Reducing workload in systematic review preparation using automated citation classification. , 2006, Journal of the American Medical Informatics Association : JAMIA.

[11]  J. Sterne,et al.  The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials , 2011, BMJ : British Medical Journal.

[12]  P. Glasziou,et al.  Systematic review automation technologies , 2014, Systematic Reviews.

[13]  S. Parekh,et al.  Assessment of the quality of reporting of randomized clinical trials in paediatric dentistry journals. , 2009, International journal of paediatric dentistry.

[14]  Stan Matwin,et al.  A new algorithm for reducing the workload of experts in performing systematic reviews , 2010, J. Am. Medical Informatics Assoc..

[15]  Philip S. Yu,et al.  Automated confidence ranked classification of randomized controlled trial articles: an aid to evidence-based medicine , 2015, J. Am. Medical Informatics Assoc..

[16]  Sophia Ananiadou,et al.  Prioritising references for systematic reviews with RobotAnalyst: A user study , 2018, Research synthesis methods.

[17]  Antonio Jimeno-Yepes,et al.  MEDLINE MeSH indexing: lessons learned from machine learning and future directions , 2012, IHI '12.

[18]  David Ogilvie,et al.  Pinpointing needles in giant haystacks: use of text mining to reduce impractical screening workload in extremely large scoping reviews , 2014, Research synthesis methods.

[19]  N. Pandis,et al.  Reporting quality of randomised controlled trials published in prosthodontic and implantology journals. , 2015, Journal of oral rehabilitation.

[20]  Guy Tsafnat,et al.  Still moving toward automation of the systematic review process: a summary of discussions at the third meeting of the International Collaboration for Automation of Systematic Reviews (ICASR) , 2019, Systematic Reviews.

[21]  J. Sterne,et al.  Assessing risk of bias in a randomized trial , 2019, Cochrane Handbook for Systematic Reviews of Interventions.

[22]  Dina Demner-Fushman,et al.  12 years on – Is the NLM medical text indexer still useful and relevant? , 2017, Journal of Biomedical Semantics.

[23]  Sophia Ananiadou,et al.  Thalia: semantic search engine for biomedical abstracts , 2018, Bioinform..

[24]  Andrew W. Brown,et al.  Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry , 2017, BMJ Open.

[25]  Carla E. Brodley,et al.  Toward modernizing the systematic review pipeline in genetics: efficient updating via data mining , 2012, Genetics in Medicine.

[26]  G. Guyatt,et al.  A practical approach to evidence-based dentistry: understanding and applying the principles of EBD. , 2014, Journal of the American Dental Association.

[27]  G. Guyatt,et al.  Comprar JAMA's Users' Guide to Medical Literature: A Manual for Evidence-Based Clinical Practice | Gordon Guyatt | 9780071590341 | Mcgraw-Hill Education , 2008 .

[28]  Muin J. Khoury,et al.  GAPscreener: An automatic tool for screening human genetic association literature in PubMed using the support vector machine technique , 2008, BMC Bioinformatics.

[29]  Carla E. Brodley,et al.  Semi-automated screening of biomedical citations for systematic reviews , 2010, BMC Bioinformatics.

[30]  Aaron M. Cohen,et al.  Optimizing Feature Representation for Automated Systematic Review Work Prioritization , 2008, AMIA.

[31]  Siddhartha R. Jonnalagadda,et al.  Automating data extraction in systematic reviews: a systematic review , 2015, Systematic Reviews.

[32]  D. Moher,et al.  Impact of the CONSORT Statement endorsement in the completeness of reporting of randomized clinical trials in restorative dentistry. , 2017, Journal of dentistry.

[33]  Lisa Hartling,et al.  Risk of bias versus quality assessment of randomised controlled trials: cross sectional study , 2009, BMJ : British Medical Journal.

[34]  Julian PT Higgins,et al.  Machine learning to assist risk-of-bias assessments in systematic reviews , 2015, International journal of epidemiology.

[35]  Byron C. Wallace,et al.  Machine learning to help researchers evaluate biases in clinical trials: a prospective, randomized user study , 2019, BMC Medical Informatics and Decision Making.

[36]  M. Page,et al.  Considering bias and conflicts of interest among the included studies , 2019, Cochrane Handbook for Systematic Reviews of Interventions.

[37]  Byron C. Wallace,et al.  Toward systematic review automation: a practical guide to using machine learning tools in research synthesis , 2019, Systematic Reviews.

[38]  Byron C. Wallace,et al.  Machine learning for identifying Randomized Controlled Trials: An evaluation and practitioner's guide , 2018, Research synthesis methods.

[39]  Andrew T. Creamer,et al.  Building an eScience Thesaurus for Librarians: A Collaboration Between the National Network of Libraries of Medicine, New England Region and an Associate Fellow at the National Library of Medicine , 2013 .

[40]  Neil R. Smalheiser,et al.  Identifying reports of randomized controlled trials (RCTs) via a hybrid machine learning and crowdsourcing approach , 2017, J. Am. Medical Informatics Assoc..

[41]  N. Pandis,et al.  The reporting quality of randomized controlled trials in orthodontics. , 2014, The journal of evidence-based dental practice.

[42]  D. Moher,et al.  CONSORT 2010 Statement: updated guidelines for reporting parallel group randomized trials , 2010, Obstetrics and gynecology.

[43]  Mei‐Ling Yeh Achieving knowledge translation in nursing care: the need for greater rigor in applying evidence to practice. , 2014, The journal of nursing research : JNR.

[44]  H. Bastian,et al.  Seventy-Five Trials and Eleven Systematic Reviews a Day: How Will We Ever Keep Up? , 2010, PLoS medicine.

[45]  Dylan Kneale,et al.  Determining the scope of the review and the questions it will address , 2019, Cochrane Handbook for Systematic Reviews of Interventions.

[46]  Byron C. Wallace,et al.  RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials , 2015, J. Am. Medical Informatics Assoc..

[47]  Pearl Brereton,et al.  Reproducibility of studies on text mining for citation screening in systematic reviews: Evaluation and checklist , 2017, J. Biomed. Informatics.

[48]  A. Rajić,et al.  A scoping review of scoping reviews: advancing the approach and enhancing the consistency , 2014, Research synthesis methods.

[49]  Aaron M. Cohen,et al.  Research Paper: Cross-Topic Learning for Work Prioritization in Systematic Review Creation and Update , 2009, J. Am. Medical Informatics Assoc..

[50]  Aaron M. Cohen,et al.  Studying the potential impact of automated document classification on scheduling a systematic review update , 2012, BMC Medical Informatics and Decision Making.

[51]  P. Ravaud,et al.  Better prioritization to increase research value and decrease waste , 2015, BMC Medicine.

[52]  Carla E. Brodley,et al.  Deploying an interactive machine learning system in an evidence-based practice center: abstrackr , 2012, IHI '12.

[53]  D. Sackett,et al.  Evidence based medicine: what it is and what it isn't , 1996, BMJ.

[54]  S. Ananiadou,et al.  Using text mining for study identification in systematic reviews: a systematic review of current approaches , 2015, Systematic Reviews.

[55]  Linda Nordling,et al.  A fairer way forward for AI in health care , 2019, Nature.

[56]  Joel D. Martin,et al.  ExaCT: automatic extraction of clinical trial characteristics from journal publications , 2010, BMC Medical Informatics Decis. Mak..

[57]  Nila A Sathe,et al.  Searching for studies: a guide to information retrieval for Campbell systematic reviews , 2017 .

[58]  Ethan M Balk,et al.  Influence of Reported Study Design Characteristics on Intervention Effect Estimates From Randomized, Controlled Trials , 2012, Annals of Internal Medicine.