The Smart environment for retrieval system evaluation — advantages and problem areas

The Smart environment provides a test-bed for implementing and evaluating a large number of different automatic search and retrieval processes. In this chapter, the basic parameters underlying the Smart system design are briefly outlined, and a comparison is made with the characteristics of more conventional retrieval systems. The principal lessons learned from the Smart experiments are described, and some of the methodological problems raised by the system design are outlined. Finally, some comments are included about the disadvantages inherent in working in the laboratory, and the insights that can be gained in such a situation.

[1]  Gerard Salton,et al.  Search strategy and the optimization of retrieval effectiveness , 1968 .

[2]  Gerard Salton,et al.  The evaluation of automatic retrieval procedures : Selected test results using the SMART system , 1965 .

[3]  Stephen E. Robertson,et al.  Relevance weighting of search terms , 1976, J. Am. Soc. Inf. Sci..

[4]  Gerard Salton,et al.  On the Specification of Term Values in Automatic Indexing , 1973 .

[5]  R. K. Waldstein,et al.  Term relevance weights in on-line information retrieval , 1977, Inf. Process. Manag..

[6]  Gerard Salton,et al.  Manipulation of trees in information retrieval , 1962, CACM.

[7]  Karen Spärck Jones.  A statistical interpretation of term specificity and its application in retrieval , 2021, J. Documentation.

[8]  Clement T. Yu,et al.  Effective Automatic Indexing Using Term Addition and Deletion , 1978, JACM.

[9]  Don R. Swanson,et al.  Searching Natural Language Text by Computer , 1960 .

[10]  C. A. Cuadra,et al.  OPENING THE BLACK BOX OF ‘RELEVANCE’ , 1967 .

[11]  Tamas E. Doszkocs,et al.  AID, an Associative Interactive Dictionary for online searching , 1978 .

[12]  Gerard Salton,et al.  Evaluation problems in interactive information retrieval , 1969, Inf. Storage Retr..

[13]  Cyril W. Cleverdon,et al.  Factors determining the performance of indexing systems , 1966 .

[14]  Michael E. Lesk,et al.  The SMART automatic document retrieval systems—an illustration , 1965, CACM.

[15]  Clement T. Yu,et al.  Precision Weighting—An Effective Automatic Indexing Method , 1976, J. ACM.

[16]  Clement T. Yu,et al.  Automatic indexing using term discrimination and term precision measurements , 1976, Information Processing & Management.

[17]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[18]  Hans Peter Luhn,et al.  A Statistical Approach to Mechanized Encoding and Searching of Literary Information , 1957, IBM J. Res. Dev..

[19]  Clement T. Yu,et al.  A theory of term importance in automatic text analysis , 1974, J. Am. Soc. Inf. Sci..

[20]  Van Rijsbergen,et al.  A theoretical basis for the use of co-occurence data in information retrieval , 1977 .

[22]  Michael E. Lesk,et al.  Computer Evaluation of Indexing and Text Processing , 1968, JACM.

[23]  Gerard Salton,et al.  Associative Document Retrieval Techniques Using Bibliographic Information , 1963, JACM.

[24]  H. P. Luhn,et al.  The automatic derivation of information retrieval encodements from machine-readable texts , 1997 .

[25]  Jean Paul Ballerini,et al.  Experiments in multilingual information retrieval using the SPIDER system , 1996, SIGIR '96.

[26]  Aviezri S. Fraenkel,et al.  Local Feedback in Full-Text Retrieval Systems , 1977, JACM.

[27]  Gerard Salton,et al.  Dynamic information and library processing , 1975 .

[28]  Tamas E. Doszkocs.  An Associative Interactive Dictionary for Online Bibliographic Searching , 1978, Jerusalem Conference on Information Technology.

[29]  Gerard Salton,et al.  On the role of words and phrases in automatic text analysis , 1975, CL.

[30]  Don R. Swanson,et al.  Machinelike indexing by people , 1962 .

[31]  Michael E. Lesk,et al.  Relevance assessments and retrieval system evaluation , 1968, Inf. Storage Retr..

[32]  Gerard Salton,et al.  A new comparison between conventional indexing (MEDLARS) and automatic text processing (SMART) , 1972, J. Am. Soc. Inf. Sci..

[33]  Carlo Vernimb,et al.  Automatic query adjustment in document retrieval , 1977, Inf. Process. Manag..

[34]  W. Bruce Croft.  Clustering large files of documents using the single-link method , 1977, J. Am. Soc. Inf. Sci..

[35]  Gerard Salton,et al.  Recent Studies in Automatic Text Analysis and Document Retrieval , 1973, JACM.

[36]  Gerard Salton,et al.  Generation and search of clustered files , 1978, TODS.