Research Paper: Clinical Decision Velocity is Increased when Meta-search Filters Enhance an Evidence Retrieval System

OBJECTIVE To test whether the use of an evidence retrieval system that uses clinically targeted meta-search filters can enhance the rate at which clinicians make correct decisions, reduce the effort involved in locating evidence, and provide an intuitive match between clinical tasks and search filters. DESIGN A laboratory experiment under controlled conditions asked 75 clinicians to answer eight randomly sequenced clinical questions, using one of two randomly assigned search engines. The first search engine Quick Clinical (QC) was equipped with meta-search filters (the combined use of meta-search and search filters) designed to answer typical clinical questions e.g., treatment, diagnosis, and the second 'library model' system (LM) offered free access to an identical evidence set with no filter support. MEASUREMENTS Changes in clinical decision making were measured by the proportion of correct post-search answers provided to questions, the time taken to answer questions, and the number of searches and links to documents followed in a search session. The intuitive match between meta-search filters and clinical tasks was measured by the proportion and distribution of filters selected for individual clinical questions. RESULTS Clinicians in the two groups performed equally well pre-search. Post search answers improved overall by 21%, with 52.2% of answers correct with QC and 54.7% with LM (chi(2) = 0.33, df = 1, p > 0.05). Users of QC obtained a significantly greater percentage of their correct answers within the first two minutes of searching compared to LM users (QC 58.2%; LM 32.9%; chi(2) = 19.203, df = 1, p < 0.001). There was a statistical difference for QC and LM survival curves, which plotted overall time to answer questions, irrespective of answer (Wilcoxon, p = 0.019) and for the average time to provide a correct answer (Wilcoxon, p = 0.006). The QC system users conducted significantly fewer searches per scenario (m = 3.0 SD = 1.15 versus m = 5.5 SD1.97, t = 6.63, df = 72, p = 0.0001). Clinicians using the QC system followed fewer document links than did those who used LM (respectively 3.9 links SD = 1.20 versus 4.7 links SD = 1.79, t = 2.13, df = 72, p = 0.0368). In 6 of the 8 questions, two meta-search filters accounted for 89% or more of clinicians' first choice, suggesting the choice of filter intuitively matched the clinical decision task at hand. CONCLUSIONS Meta-search filters result in clinicians arriving at answers more quickly than unconstrained searches across information sources, and appear to increase the rate with which correct decisions are made. In time restricted clinical settings meta-search filters may thus improve overall decision accuracy, as fewer searches that could otherwise lead to a correct answer are abandoned. Meta-search filters appear to be intuitive to use, suggesting that the simplicity of the user model would fit very well into clinical settings.

[1]  R Smith,et al.  Information in practice , 1996, BMJ.

[2]  E Coiera,et al.  Maximising the uptake of evidence into clinical practice: an information economics approach , 2001, The Medical journal of Australia.

[3]  J Benichou,et al.  A study comparing centralized CD-ROM and decentralized intranet access to MEDLINE. , 2000, Bulletin of the Medical Library Association.

[4]  Lucas M. Bachmann,et al.  Research Paper: Identifying Diagnostic Studies in MEDLINE: Reducing the Number Needed to Read , 2002, J. Am. Medical Informatics Assoc..

[5]  Johanna I. Westbrook,et al.  Do online information retrieval systems help experienced clinicians answer clinical questions? , 2005, Journal of the American Medical Informatics Association : JAMIA.

[6]  Johanna I. Westbrook,et al.  Research Paper: Do clinicians use online evidence to support patient care? a study of 55, 000 clinicians , 2003, J. Am. Medical Informatics Assoc..

[7]  Johanna I Westbrook,et al.  The Impact of an Online Evidence System on Confidence in Decision Making in a Controlled Setting , 2005, Medical decision making : an international journal of the Society for Medical Decision Making.

[8]  I. Scott,et al.  The use of evidence-based medicine in the practice of consultant physicians. Results of a questionnaire survey. , 2000, Australian and New Zealand journal of medicine.

[9]  D. Covell,et al.  Information needs in office practice: are they being met? , 1985, Annals of internal medicine.

[10]  W Pratt,et al.  Physician's information customizer (PIC): using a shareable user model to filter the medical literature. , 1995, Medinfo. MEDINFO.

[11]  K. Ramos,et al.  Real-time information-seeking behavior of residency physicians. , 2003, Family medicine.

[12]  M. Ebell,et al.  Obstacles to answering doctors' questions about patient care with evidence: qualitative study , 2002, BMJ : British Medical Journal.

[13]  K. Schwartz,et al.  Use of on-line evidence-based resources at the point of care. , 2003, Family medicine.

[14]  Richard Smith What clinical information do doctors need? , 1996, BMJ.

[15]  Enrico Coiera,et al.  Architecture for Knowledge-Based and Federated Search of Online Clinical Evidence , 2005, Journal of medical Internet research.

[16]  Linda A. Watson,et al.  Information Retrieval: A Health and Biomedical Perspective. , 2005 .

[17]  B. J. Ingui,et al.  Searching for clinical prediction rules in MEDLINE. , 2001, Journal of the American Medical Informatics Association : JAMIA.

[18]  D. Sackett,et al.  Finding and applying evidence during clinical rounds: the "evidence cart". , 1998, JAMA.

[19]  M. Ebell,et al.  Analysis of questions asked by family doctors regarding patient care , 1999, BMJ.

[20]  Yindalon Aphinyanagphongs,et al.  Research Paper: Text Categorization Models for High-Quality Article Retrieval in Internal Medicine , 2004, J. Am. Medical Informatics Assoc..

[21]  R. Brian Haynes,et al.  Developing optimal search strategies for detecting clinically sound studies in MEDLINE. , 1994, Journal of the American Medical Informatics Association : JAMIA.

[22]  R. Haynes,et al.  Medline : analytical survey scientifically strong studies of diagnosis from Optimal search strategies for retrieving , 2004 .

[23]  Farah Magrabi,et al.  General practitioners' use of online evidence during consultations , 2005, Int. J. Medical Informatics.