The challenge of high recall in biomedical systematic search

Clinical systematic reviews are based on expert, laborious search of well-annotated literature. Boolean search on bibliographic databases, such as MEDLINE, continues to be the preferred discovery method, but the size of these databases, now approaching 20 million records, makes it impossible to fully trust these searching methods. We are investigating the trade-offs between Boolean and ranked retrieval. Our findings show that although Boolean search has limitations, it is not obvious that ranking is superior, and illustrate that a single query cannot be used to resolve an information need. Our experiments show that a combination of less complicated Boolean queries and ranked retrieval outperforms either of them individually, leading to possible time savings over the current process.

[1]  K. Dickersin,et al.  Systematic Reviews: Identifying relevant studies for systematic reviews , 1994 .

[2]  David Moher,et al.  No consensus exists on search reporting methods for systematic reviews. , 2008, Journal of clinical epidemiology.

[3]  R. Brian Haynes,et al.  Developing optimal search strategies for detecting clinically sound studies in MEDLINE. , 1994, Journal of the American Medical Informatics Association : JAMIA.

[4]  H. Handoll,et al.  Lessons for search strategies from a systematic review, in The Cochrane Library, of nutritional supplementation trials in patients after hip fracture. , 2001, The American journal of clinical nutrition.

[5]  Alistair Moffat,et al.  Has adhoc retrieval improved since 1994? , 2009, SIGIR.

[6]  R. Haynes,et al.  Optimal search strategies for retrieving systematic reviews from Medline: analytical survey , 2004, BMJ : British Medical Journal.

[7]  J. McGowan,et al.  Systematic reviews need systematic searchers. , 2005, Journal of the Medical Library Association : JMLA.

[8]  K. A. McKibbon,et al.  Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey , 2005, BMJ : British Medical Journal.

[9]  J. McGowan,et al.  Errors in search strategies were identified by type and frequency. , 2006, Journal of clinical epidemiology.

[10]  William R. Hersh,et al.  Reducing workload in systematic review preparation using automated citation classification. , 2006, Journal of the American Medical Informatics Association : JAMIA.

[11]  Edward A. Fox,et al.  Research Contributions , 2014 .

[12]  Stephen E. Robertson,et al.  Okapi at TREC-6 Automatic ad hoc, VLC, routing, filtering and QSDR , 1997, TREC.

[13]  Joon Ho Lee,et al.  Properties of extended Boolean models in information retrieval , 1994, SIGIR '94.

[14]  Li Zhang,et al.  Optimizing search strategies to identify randomized controlled trials in MEDLINE , 2006, BMC medical research methodology.