The NICE UK geographic search filters for MEDLINE and Embase (Ovid): Post‐development study to further evaluate precision and number‐needed‐to‐read when retrieving UK evidence

BACKGROUND The National Institute for Health and Care Excellence's (NICE) United Kingdom (UK) geographic search filters for MEDLINE and Embase (OVID) retrieve evidence in literature searches for UK-focused research topics with high recall. Their precision and number-needed-to-read (NNR) was examined previously in case studies using a single review. This paper details a larger post-development study that was conducted to test the NICE UK filters' precision and NNR more extensively. METHODS The filters' recall of included UK references from 100 reviews was calculated. As reproducible search strategies were not available for every review, the MEDLINE filter's precision and NNR were calculated using strategies from 25 reviews. Strategies from nine reviews were used for the Embase filter. RESULTS The MEDLINE filter achieved an average of 96.4% recall for the included UK references from the 100 reviews and the Embase filter achieved an average of 97.4% recall. Compared to not using a filter, the MEDLINE filter achieved an average of 98.9% recall for the 25 reviews. Precision was increased by an average of 7.8 times, reducing the NNR from 357 to 46. The Embase filter achieved an average of 97.1% recall for the nine reviews. Precision was increased by an average of 5.1 times, reducing the NNR from 746 to 146. CONCLUSION There is more evidence to demonstrate that the NICE UK filters retrieve the majority of UK evidence from MEDLINE and Embase while increasing precision and reducing NNR. The filters can save time spent on selecting evidence for UK-focused research topics.

[1]  D. Moher,et al.  An alternative to the hand searching gold standard: validating methodological search filters using relative recall , 2006, BMC medical research methodology.

[2]  Jenny Craven,et al.  The MEDLINE UK filter: development and validation of a geographic search filter to retrieve research about the UK from OVID MEDLINE , 2017, Health information and libraries journal.

[3]  R. Milne,et al.  Promoting physical activity in South Asian Muslim women through "exercise on prescription". , 2002, Health technology assessment.

[4]  Jenny Craven,et al.  How to Develop a Validated Geographic Search Filter: Five Key Steps , 2020 .

[5]  N. Siegfried,et al.  Developing a geographic search filter to identify randomised controlled trials in Africa: finding the optimal balance between sensitivity and precision. , 2011, Health information and libraries journal.

[6]  A. Mabrook,et al.  Can nurse practitioners offer a quality service? An evaluation of a year's work of a nurse led minor injury unit. , 1998, Journal of accident & emergency medicine.

[7]  P. Shekelle,et al.  Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement , 2015, Systematic Reviews.

[8]  Dorothy L. Espelage,et al.  Best practice guidelines for abstract screening large‐evidence systematic reviews and meta‐analyses , 2019, Research Synthesis Methods.

[9]  J. Schwartz,et al.  Mortality risk attributable to high and low ambient temperature: a multicountry observational study , 2015, The Lancet.

[10]  Jenny Craven,et al.  The Embase UK filter: validation of a geographic search filter to retrieve research about the UK from OVID Embase. , 2019, Health information and libraries journal.

[11]  W. H. Wilkinson,et al.  Dietary and other characteristics relevant for coronary heart disease in men of Indian, West Indian and European descent in London. , 1988, Atherosclerosis.

[12]  A. Mann,et al.  Formal and informal care for people with dementia: Factors associated with service receipt , 2002, Aging & mental health.

[13]  J. McGowan,et al.  PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. , 2016, Journal of clinical epidemiology.

[14]  M. Marmot,et al.  Relation of central obesity and insulin resistance with high diabetes prevalence and cardiovascular risk in South Asians , 1991, The Lancet.

[15]  Tamara Rader,et al.  Searching for and selecting studies , 2019, Cochrane Handbook for Systematic Reviews of Interventions.

[16]  V. Costarelli,et al.  Attitudes towards health and exercise of overweight women , 2008, The journal of the Royal Society for the Promotion of Health.

[17]  M. Marmot,et al.  Diabetes, hyperinsulinaemia, and coronary risk factors in Bangladeshis in east London. , 1988, British heart journal.

[18]  M. Strippoli,et al.  Parental understanding of wheeze and its impact on asthma prevalence estimates , 2006, European Respiratory Journal.

[19]  Liz Payne,et al.  So many filters, so little time: the development of a search filter appraisal checklist. , 2008, Journal of the Medical Library Association : JMLA.

[20]  K. Hawton,et al.  Ethnic differences in self-harm, rates, characteristics and service provision: three-city cohort study. , 2010, The British journal of psychiatry : the journal of mental science.

[21]  M. Marmot,et al.  Ethnic variation in childhood asthma and wheezing illnesses: findings from the Millennium Cohort Study. , 2007, International journal of epidemiology.

[22]  J. Krska,et al.  Involving the public and other stakeholders in development and evaluation of a community pharmacy alcohol screening and brief advice service. , 2014, Public health.

[23]  Lucas M. Bachmann,et al.  Research Paper: Identifying Diagnostic Studies in MEDLINE: Reducing the Number Needed to Read , 2002, J. Am. Medical Informatics Assoc..

[24]  C. Gilleard,et al.  The effect of inpatient respite care on elderly patients and their carers. , 1994, Age and ageing.

[25]  I. Durand-zaleski,et al.  Cost‐effectiveness of systemic and regional chemotherapy for the treatment of patients with unresectable colorectal liver metastases , 1998, Cancer.

[26]  Rosario Arquero-Avilés,et al.  Errors in search strategies used in systematic reviews and their effects on information retrieval , 2019, Journal of the Medical Library Association : JMLA.

[27]  J. McGowan,et al.  PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation , 2018, Annals of Internal Medicine.

[28]  Melissa L Rethlefsen,et al.  Reproducibility of Search Strategies Is Poor in Systematic Reviews Published in High-Impact Pediatrics, Cardiology and Surgery Journals: A Cross-Sectional Study , 2016, PloS one.

[29]  M. Marmot,et al.  Differences in biological risk factors for cardiovascular disease between three ethnic groups in the Whitehall II study. , 1999, Atherosclerosis.

[30]  Joan Mendivil,et al.  [Development of a geographic filter for PubMed to identify studies performed in Spain]. , 2006, Revista espanola de cardiologia.