Indexing and retrieval performance: the logical evidence

This article presents a logical analysis of the characteristics of indexing and their effects on retrieval performance. It establishes the ability to ask the questions one needs to ask as the foundation of performance evaluation, and recall and discrimination as the basic quantitative performance measures for binary noninteractive retrieval systems. It then defines the characteristics of indexing that affect retrieval—namely, indexing devices, viewpoint-based and importance-based indexing exhaustivity, indexing specificity, indexing correctness, and indexing consistency—and examines in detail their effects on retrieval. It concludes that retrieval performance depends chiefly on the match between indexing and the requirements of the individual query and on the adaptation of the query formulation to the characteristics of the retrieval system, and that the ensuing complexity must be considered in the design and testing of retrieval systems. © 1994 John Wiley & Sons, Inc.

[1]  William S. Cooper,et al.  On selecting a measure of retrieval effectiveness , 1973, J. Am. Soc. Inf. Sci..

[2]  Donald W. King,et al.  The Evaluation of Information Services and Products , 1971 .

[3]  William S. Cooper,et al.  A definition of relevance for information retrieval , 1971, Inf. Storage Retr..

[4]  A. M. Pejtersen,et al.  Design and test of a database for fiction, based on an analysis of children's search behavior , 1986 .

[5]  Janet L. Kolodner,et al.  Indexing and Retrieval , 1993 .

[6]  A. M. Pejtersen,et al.  Fiction Retrieval: Experimental Design and Evaluation of a Search System based on Users' Value criteria (Part 2) , 1983, J. Documentation.

[7]  John A. Swets,et al.  Effectiveness of information retrieval methods , 1969 .

[8]  Marcia J. Bates,et al.  The design of browsing and berrypicking techniques for the online search interface , 1989 .

[9]  Stephen E. Robertson,et al.  THE PARAMETRIC DESCRIPTION OF RETRIEVAL TESTS: PART I: THE BASIC PARAMETERS , 1969 .

[10]  I. J. Good,et al.  The decision-theory approach to the evaluation of information-retrieval systems , 1967, Inf. Storage Retr..

[11]  William S. Cooper,et al.  Indexing documents by gedanken experimentation , 1978, J. Am. Soc. Inf. Sci..

[12]  D. A. Kemp.  Relevance, pertinence and information system development , 1974, Inf. Storage Retr..

[13]  Barbara A. Montague,et al.  Testing, comparison, and evaluation of recall, relevance, and cost of coordinate indexing with links and roles , 1965 .

[14]  J A Swets,et al.  Information Retrieval Systems. , 1963, Science.

[15]  Dagobert Soergel,et al.  Organizing information - principles of data base and retrieval systems , 1985 .

[16]  William Goffman,et al.  A methodology for test and evaluation of information retrieval systems , 1966, Inf. Storage Retr..

[17]  M. E. Maron.  Depth of indexing , 1979, J. Am. Soc. Inf. Sci..

[18]  William S. Cooper,et al.  On selecting a measure of retrieval effectiveness part II. Implementation of the philosophy , 1973, J. Am. Soc. Inf. Sci..