Use of Recommended Search Strategies in Systematic Reviews and the Impact of Librarian Involvement: A Cross-Sectional Survey of Recent Authors

Background Previous research looking at published systematic reviews has shown that their search strategies are often suboptimal and that librarian involvement, though recommended, is low. Confidence in the results, however, is limited due to poor reporting of search strategies the published articles. Objectives To more accurately measure the use of recommended search methods in systematic reviews, the levels of librarian involvement, and whether librarian involvement predicts the use of recommended methods. Methods A survey was sent to all authors of English-language systematic reviews indexed in the Database of Abstracts of Reviews of Effects (DARE) from January 2012 through January 2014. The survey asked about their use of search methods recommended by the Institute of Medicine, Cochrane Collaboration, and the Agency for Healthcare Research and Quality and if and how a librarian was involved in the systematic review. Rates of use of recommended methods and librarian involvement were summarized. The impact of librarian involvement on use of recommended methods was examined using a multivariate logistic regression. Results 1560 authors completed the survey. Use of recommended search methods ranged widely from 98% for use of keywords to 9% for registration in PROSPERO and were generally higher than in previous studies. 51% of studies involved a librarian, but only 64% acknowledge their assistance. Librarian involvement was significantly associated with the use of 65% of recommended search methods after controlling for other potential predictors. Odds ratios ranged from 1.36 (95% CI 1.06 to 1.75) for including multiple languages to 3.07 (95% CI 2.06 to 4.58) for using controlled vocabulary. Conclusions Use of recommended search strategies is higher than previously reported, but many methods are still under-utilized. Librarian involvement predicts the use of most methods, but their involvement is under-reported within the published article.

[1]  J. Higgins.  Cochrane handbook for systematic reviews of interventions. Version 5.1.0 [updated March 2011]. The Cochrane Collaboration , 2011 .

[2]  F. Boop,et al.  Methodology and reporting of meta-analyses in the neurosurgical literature. , 2014, Journal of neurosurgery.

[3]  D. Moher,et al.  An Evaluation of Epidemiological and Reporting Characteristics of Complementary and Alternative Medicine (CAM) Systematic Reviews (SRs) , 2013, PloS one.

[4]  Paul Fehrmann,et al.  Comprehensive computer searches and reporting in systematic reviews , 2011, Research synthesis methods.

[5]  David Moher,et al.  Network meta-analyses could be improved by searching more sources and by involving a librarian. , 2014, Journal of clinical epidemiology.

[6]  Maureen Dobbins,et al.  An optimal search filter for retrieving systematic reviews and meta-analyses , 2012, BMC Medical Research Methodology.

[7]  P. Major,et al.  Search and selection methodology of systematic reviews in orthodontics (2000-2004). , 2006, American journal of orthodontics and dentofacial orthopedics : official publication of the American Association of Orthodontists, its constituent societies, and the American Board of Orthodontics.

[8]  K. Ann McKibbon,et al.  Systematic Reviews and Librarians , 2006, Libr. Trends.

[9]  N. Mohaghegh,et al.  WHY THE IMPACT FACTOR OF JOURNALS SHOULD NOT BE USED FOR EVALUATING RESEARCH , 2005 .

[10]  U. Nassar,et al.  Methodological quality and descriptive characteristics of prosthodontic-related systematic reviews. , 2013, Journal of oral rehabilitation.

[11]  Melissa L Rethlefsen,et al.  Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. , 2015, Journal of clinical epidemiology.

[12]  Catherine Sherrington,et al.  Cochrane reviews used more rigorous methods than non-Cochrane reviews: survey of systematic reviews in physiotherapy. , 2009, Journal of clinical epidemiology.

[13]  J. Ioannidis,et al.  The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. , 2009, Journal of clinical epidemiology.

[14]  Nancy H Tannery,et al.  Reproducibility of Literature Search Reporting in Medical Education Reviews , 2011, Academic medicine : journal of the Association of American Medical Colleges.

[15]  D. Moher,et al.  Systematic reviews involving complementary and alternative medicine interventions had higher quality of reporting than conventional medicine reviews. , 2005, Journal of clinical epidemiology.

[16]  J. McGowan,et al.  Systematic reviews need systematic searchers. , 2005, Journal of the Medical Library Association : JMLA.

[17]  Su Golder,et al.  Some improvements are apparent in identifying adverse effects in systematic reviews from 1994 to 2011. , 2013, Journal of clinical epidemiology.

[18]  David Moher,et al.  Epidemiology and Reporting Characteristics of Systematic Reviews , 2007, PLoS medicine.

[19]  David Moher,et al.  An evidence-based practice guideline for the peer review of electronic search strategies. , 2009, Journal of clinical epidemiology.

[20]  Melissa L Rethlefsen,et al.  Engaging medical librarians to improve the quality of review articles. , 2014, JAMA.

[21]  Linda C. Li,et al.  Survey of the Reporting Characteristics of Systematic Reviews in Rehabilitation , 2013, Physical Therapy.

[22]  S. Warren,et al.  Survey of systematic review authors in dentistry: challenges in methodology and reporting. , 2009, Journal of dental education.

[23]  Su Golder,et al.  Comparison of search strategies in systematic reviews of adverse effects to other systematic reviews. , 2014, Health information and libraries journal.

[24]  A. Weller,et al.  Mounting evidence that librarians are essential for comprehensive literature searches for meta-analyses and Cochrane reports. , 2004, Journal of the Medical Library Association : JMLA.

[25]  L. Ge,et al.  The Assessment of the Quality of Reporting of Systematic Reviews/Meta-Analyses in Diagnostic Tests Published by Authors in China , 2014, PloS one.

[26]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement , 2009, BMJ.

[27]  L. Maggio,et al.  The role of medical librarians in medical education review articles. , 2012, Journal of the Medical Library Association : JMLA.

[28]  P. Glasziou,et al.  Are systematic reviews up-to-date at the time of publication? , 2013, Systematic Reviews.

[29]  Howard Balshem,et al.  Finding Evidence for Comparing Medical Interventions , 2011 .

[30]  S. Shakir,et al.  Systematic reviews of adverse effects of drug interventions: a survey of their conduct and reporting quality , 2009, Pharmacoepidemiology and drug safety.

[31]  Li Zhang,et al.  Reporting of the Role of the Expert Searcher in Cochrane Reviews , 2006 .

[32]  Laura A. Levit,et al.  Finding what works in health care : standards for systematic reviews , 2011 .

[33]  S. Papageorgiou,et al.  Reporting characteristics of meta-analyses in orthodontics: methodological assessment and statistical recommendations. , 2014, European journal of orthodontics.

[34]  S. Boccia,et al.  Evaluation of the Endorsement of the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) Statement on the Quality of Published Systematic Review and Meta-Analyses , 2013, PloS one.

[35]  J. McGowan,et al.  Errors in search strategies were identified by type and frequency. , 2006, Journal of clinical epidemiology.

[36]  C. Faggion,et al.  Search strategies in systematic reviews in periodontology and implant dentistry. , 2013, Journal of clinical periodontology.

[37]  Sebastiaan L. Knijnenburg,et al.  Quality of systematic reviews in pediatric oncology--a systematic review. , 2009, Cancer treatment reviews.

[38]  N. Pandis,et al.  Systematic reviews published in higher impact clinical journals were of higher quality. , 2014, Journal of clinical epidemiology.

[39]  P. Major,et al.  An evaluation of search and selection methods used in dental systematic reviews published in English. , 2006, Journal of the American Dental Association.

[40]  Adriana Yoshii,et al.  Analysis of the reporting of search strategies in Cochrane systematic reviews. , 2009, Journal of the Medical Library Association : JMLA.

[41]  P. Major,et al.  Benchmarking of reported search and selection methods of systematic reviews by dental speciality , 2007, Evidence-Based Dentistry.

[42]  Steve Halligan,et al.  Systematic reviews of diagnostic tests in cancer: review of methods and reporting , 2006, BMJ : British Medical Journal.

[43]  David Moher,et al.  From QUOROM to PRISMA: A Survey of High-Impact Medical Journals' Instructions to Authors and a Review of Systematic Reviews in Anesthesia Literature , 2011, PloS one.

[44]  S. Golder,et al.  Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects. , 2008, Journal of clinical epidemiology.

[45]  I. Olkin,et al.  Meta-analysis of observational studies in epidemiology - A proposal for reporting , 2000 .

[46]  David Moher,et al.  No consensus exists on search reporting methods for systematic reviews. , 2008, Journal of clinical epidemiology.