Spatial querying for image retrieval: a user-oriented evaluation

Epic is an image retrieval system that implements a novel spatial-querying mechanism. A usercentred, task-oriented, comparative evaluation of Epic was undertaken in which two versions of the system--one set up to enable spatial queries only, the other allowing textual queries only-were compared. Use was made of the two systems by design professionals in simulated work task situations, and quantitative and qualitative data collected as indicators of the levels of users’ satisfaction. Re sults demonstrated that users often had a ‘mental image’ of a potentially satisfying picture in mind, that they were happy to express this need in visual terms, and that in doing so they preferred to have access to Epic’s spatialquerying facility. Success in obtaining statistically significant results appears to support validation of the novel methodological framework adopted.

[1]  Vijay V. Raghavan,et al.  Modeling and retrieving images by content , 1997, Inf. Process. Manag..

[2]  Alberto Del Bimbo,et al.  Visual information retrieval , 1999 .

[3]  Julia Galliers,et al.  Evaluating natural language processing systems , 1995 .

[4]  Donna K. Harman,et al.  Evaluation Issues in Information Retrieval , 1992, Inf. Process. Manag..

[5]  Nicholas J. Belkin,et al.  Evaluation of a tool for visualization of information retrieval results , 1996, SIGIR '96.

[6]  Cyril W. Cleverdon,et al.  Factors determining the performance of indexing systems , 1966 .

[7]  Stephen A. R. Scrivener,et al.  Developing practice with theory in HCI: applying models of spatial cognition for the design of pictorial databases , 1996, Int. J. Hum. Comput. Stud..

[8]  Jean Tague-Sutcliffe.  Some perspectives on the evaluation of information retrieval systems , 1996 .

[9]  Vijay V. Raghavan,et al.  Content-Based Image Retrieval Systems - Guest Editors' Introduction , 1995, Computer.

[10]  Stefano Mizzaro,et al.  Evaluating user interfaces to information retrieval systems: a case study on user support , 1996, SIGIR '96.

[11]  Stephen P. Harter,et al.  Research Methods in Librarianship: Techniques and Interpretation , 1980 .

[12]  Jean Tague-Sutcliffe,et al.  The Pragmatics of Information Retrieval Experimentation Revisited , 1997, Inf. Process. Manag..

[13]  Joemon M. Jose,et al.  A Retrieval Mechanism for Semi-Structured Photographic Collections , 1997, DEXA.

[14]  David J. HarperThe.  Epic: a Photograph Retrieval System Based on Evidence Combination Approach , 1998 .

[15]  William R. Hersh,et al.  A task-oriented approach to information retrieval evaluation , 1996 .

[16]  William R. Hersh,et al.  Relevance and Retrieval Evaluation: Perspectives from Medicine , 1994, J. Am. Soc. Inf. Sci..

[17]  Peter G. B. Enser.  Pictorial information retrieval , 1995 .

[18]  Estelle Brodman,et al.  Research Methods in Librarianship; Techniques and Interpretation. , 1981 .

[19]  Tefko Saracevic,et al.  Evaluation of evaluation in information retrieval , 1995, SIGIR '95.

[20]  Stephen E. Robertson,et al.  Evaluating Interactive Systems in TREC , 1996, J. Am. Soc. Inf. Sci..

[21]  William R. Hersh.  Relevance and retrieval evaluation: perspectives from medicine , 1994 .

[22]  Margaret King,et al.  Evaluating natural language processing systems , 1996, CACM.

[23]  Peter Ingwersen,et al.  The development of a method for the evaluation of interactive information retrieval systems , 1997, J. Documentation.

[24]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.