A Comparison of Two Methods for Indexing and Retrieval from a Full-text Medical Database

The objective of this study was to compare how well medical professionals are able to retrieve relevant literature references using two computerized literature searching systems that provide automated (non-human) indexing of content. The first program was SAPHIRE, which features concept-based indexing, free-text input of queries, and ranking of retrieved references for relevance. The second program was SWORD, which provides single-word searching using Boolean operators (AND, OR). Sixteen fourth-year medical students partici pated in the study. The database for searching was six volumes from the 1989 Yearbook series. The queries were ten questions generated on teaching rounds. All subjects searched half the queries with each program. After the searching, each subject was given a ques tionnaire about prior experience and preferences about the two programs. Recall (proportion of relevant articles retrieved from the database) and precision (proportion of relevant articles in the retrieved set) were measured for each search done by each participant. Mean recall was 57.6% with SAPHIRE; it was 58.6% with SWORD. Precision was 48.1% with SAPHIRE VS 57.6% with SWORD. Each program was rated easier to use than the other by half of the searchers, and preferences were associated with better searching performance for that program. Both systems achieved recall and precision comparable to existing systems and may represent effective alternatives to MEDLINE and other retrieval systems based on human indexing for searching medical literature. Key words: information retrieval systems; auto mated indexing; medical literature; MEDLINE. (Med Decis Making 1993;13:220-226)

[1]  K. A. McKibbon,et al.  Online access to medline in clinical settings , 2020 .

[2]  Christine L. Borgman,et al.  Why are online catalogs hard to use? Lessons learned from information-retrieval studies , 1986, J. Am. Soc. Inf. Sci..

[3]  D Slingluff,et al.  An end user search service in an academic health sciences library. , 1985, Medical reference services quarterly.

[4]  Christine L. Borgman,et al.  Why are Online Catalogs Hard to Use? Lessons Learned from Information=Retrieval Studies , 1986 .

[5]  C J Walker,et al.  A review of medical education and medical informatics , 1989, Academic medicine : journal of the Association of American Medical Colleges.

[6]  Tefko Saracevic,et al.  Individual Differences in Organizing, Searching and Retrieving Information. , 1991 .

[7]  B. Buchanan,et al.  Physicians' information needs: analysis of questions posed during clinical teaching. , 1991, Annals of internal medicine.

[8]  Martha Kirby Ms,et al.  MEDLINE Searching on Colleague , 1986 .

[9]  E. Shortliffe.  Computer programs to support clinical decision making. , 1990, JAMA.

[10]  J W Williamson,et al.  Health science information management and continuing education of physicians. A survey of U.S. primary care practitioners and their opinion leaders. , 1989, Annals of internal medicine.

[11]  W R Hersh.  Evaluation of Meta-1 for a Concept-based Approach to the Automated Indexing and Retrieval of Bibliographic and Full-text Databases , 1991, Medical decision making : an international journal of the Society for Medical Decision Making.

[12]  K. A. McKibbon,et al.  Online access to MEDLINE in clinical settings. A study of use and usefulness. , 1990, Annals of internal medicine.

[13]  D A Evans,et al.  Automatic Indexing of Abstracts via Natural-language Processing Using a Simple Thesaurus , 1991, Medical decision making : an international journal of the Society for Medical Decision Making.

[14]  M E Funk,et al.  Indexing consistency in MEDLINE. , 1983, Bulletin of the Medical Library Association.

[15]  D. Covell,et al.  Information needs in office practice: are they being met? , 1985, Annals of internal medicine.

[16]  William R. Hersh,et al.  A Comparative Analysis of Retrieval Effectiveness for Three Methods of Indexing AIDS-Related Abstracts. , 1991 .

[17]  R A Greenes,et al.  SAPHIRE--an information retrieval system featuring concept matching, automatic indexing, probabilistic retrieval, and hierarchical relationships. , 1990, Computers and biomedical research, an international journal.