Time, relevance and interaction modelling for information retrieval

The most common method for assessing the worth of an information retrieval (lR) system is through precision and recall graphs. These graphs show how precise an IR engine is when working at fixed levels of recall. This paper introduces number-lo-view graphs, a new graphing method based on an early evaluation measure, which supplement precisionrecall graphs by plotting the number of relevant documents a user wishes against the number of documents they would have to view to encounter them. The paper also proposes a step forward from number-to-view graphs that directly includes presentation, interface and temporal issues within the same frame work as engine effectiveness: time-to-view graphs. Taken together, these graphs and models introduce a new evaluation approach called Expected Search Duration.

[1]  David Benyon,et al.  The skull beneath the skin: entity-relationship models of information artifacts , 1996, Int. J. Hum. Comput. Stud..

[2]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[3]  Donna K. Harman,et al.  Overview of the Fourth Text REtrieval Conference (TREC-4) , 1995, TREC.

[4]  Stuart Card,et al.  User technology—from pointing to pondering , 1986, HPW '86.

[5]  Mark D. Dunlop,et al.  Hypermedia. And probabilistic retrieval , 1991, RIAO.

[6]  Stephen E. Robertson,et al.  Large Test Collection Experiments on an Operational, Interactive System: Okapi at TREC , 1995, Inf. Process. Manag..

[7]  W. Buxton.  Human-Computer Interaction , 1988, Springer Berlin Heidelberg.

[8]  Michael E. Senko,et al.  Information Storage and Retrieval Systems , 1969 .

[9]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[10]  Mark D. Dunlop,et al.  Hypermedia and Free Text Retrieval , 1993, Inf. Process. Manag..

[11]  P MoranThomas,et al.  The keystroke-level model for user performance time with interactive systems , 1980 .

[12]  M. F. Porter,et al.  An algorithm for suffix stripping , 1997 .

[13]  Allen Newell,et al.  The keystroke-level model for user performance time with interactive systems , 1980, CACM.

[14]  S. Mizzaro,et al.  A Cognitive Analysis of Information Retrieval , 1996 .

[15]  Jock D. Mackinlay,et al.  The cost-of-knowledge characteristic function: display evaluation for direct-walk dynamic information visualizations , 1994, CHI Conference Companion.

[16]  W. S. Cooper.  Expected search length: A single measure of retrieval effectiveness based on the weak ordering action of retrieval systems , 1968 .

[17]  Mark Sanderson,et al.  NRT: News Retrieval Tool , 1991, Electron. Publ..

[18]  Donna Harman.  User-friendly systems instead of user-friendly front-ends , 1997 .

[19]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[20]  Donna K. Harman,et al.  User-Friendly Systems Instead of User-Friendly Front-Ends , 1997, J. Am. Soc. Inf. Sci..

[21]  Maristella Agosti,et al.  Information Retrieval and Hypertext , 1996, Information Retrieval and Hypertext.