Iterative exploration, design and evaluation of support for query reformulation in interactive information retrieval

We report on the progressive investigation of techniques for supporting interactive query reformulation in the TREC Interactive Track. Two major issues were explored over four successive years: methods of term suggestion; and, interface design to support different system functionalities. Each year's results led to the following year's investigation, with respect to both of these issues. This paper presents first the general motivation for the entire series of studies; then an overview of each year's investigation, its results, and how they influenced the next year's investigation. We discuss what has been learned through this series of investigations about effective term suggestion, usable and useful interface design, and the relationships between these two in support of the TREC Interactive Track task. We conclude with comments about the general methodology employed over this series of studies, and its relevance to the development and evaluation of interactive information retrieval systems.

[1]  B. Shneiderman,et al.  The dynamic HomeFinder: evaluating dynamic queries in a real-estate information exploration system , 1992, SIGIR '92.

[2]  Donald E. Walker.  Interactive bibliographic search : the user/computer interface , 1971 .

[3]  Marti A. Hearst,et al.  Cat-a-Cone: an interactive interface for specifying searches and viewing retrieval results using a large category hierarchy , 1997, SIGIR '97.

[4]  Gerard Salton,et al.  Improving retrieval performance by relevance feedback , 1997, J. Am. Soc. Inf. Sci..

[5]  Nicholas J. Belkin,et al.  Rutgers' TREC 2001 Interactive Track Experience , 2001, TREC.

[6]  Nicholas J. Belkin,et al.  Relevance feedback: usage, usability, utility , 1996 .

[7]  Gerard Salton,et al.  The SMART Retrieval System—Experiments in Automatic Document Processing , 1971 .

[8]  James Allan,et al.  Aspect windows, 3-D visualizations, and indirect comparisons of information retrieval systems , 1998, SIGIR '98.

[9]  Gary Marchionini,et al.  Design of Interfaces for Information Seeking. , 1998 .

[10]  Donna Harman,et al.  Information Processing and Management , 2022 .

[11]  Karen Sparck Jones,et al.  Okapi at TREC{7: automatic ad hoc, ltering, VLC and interactive track , 1999 .

[12]  J. J. Rocchio,et al.  Relevance feedback in information retrieval , 1971 .

[13]  Robert M. Losee,et al.  Feedback in Information Retrieval. , 1996 .

[14]  Nicholas J. Belkin,et al.  Rutgers Interactive Track at TREC-5 , 1996, TREC.

[15]  Lokman I. Meho,et al.  IRIS at TREC-7 , 1998, TREC.

[16]  Nicholas J. Belkin,et al.  Rutgers' TREC-6 Interactive Track Experience , 1997, TREC.

[17]  Edward A. Fox,et al.  Users, User Interfaces, and Objects: Envision, a Digital Library , 1993, J. Am. Soc. Inf. Sci..

[18]  Paul Over,et al.  The TREC interactive track: an annotated bibliography , 2001, Inf. Process. Manag..

[19]  Donna K. Harman,et al.  Relevance feedback revisited , 1992, SIGIR '92.

[20]  B. Shneiderman.  Designing the User Interface (3rd Ed.) , 1998 .

[21]  Ben Shneiderman,et al.  The Psychology of Menu Selection: Designing Cognitive Control at the Human/Computer Interface , 1991 .

[22]  Peter Ingwersen,et al.  The development of a method for the evaluation of interactive information retrieval systems , 1997, J. Documentation.

[23]  W. Bruce Croft,et al.  The INQUERY Retrieval System , 1992, DEXA.

[24]  Ben Shneiderman,et al.  Designing The User Interface , 2013 .

[25]  Nicholas J. Belkin,et al.  Relevance Feedback versus Local Context Analysis as Term Suggestion Devices: Rutgers' TREC-8 Interactive Track Experience , 1999, TREC.

[26]  Louis M. Gomez,et al.  Formative design evaluation of superbook , 1989, TOIS.

[27]  Stephen E. Robertson,et al.  Okapi at TREC-7: Automatic Ad Hoc, Filtering, VLC and Interactive , 1998, TREC.

[28]  Nicholas J. Belkin,et al.  New tools and old habits : the interactive searching behavior of expert online searchers using INQUERY , 1994 .

[29]  W. Bruce Croft,et al.  Query expansion using local and global document analysis , 1996, SIGIR '96.

[30]  Nicholas J. Belkin,et al.  Using Relevance Feedback and Ranking in Interactive Searching , 1995, TREC.

[31]  Nicholas J. Belkin,et al.  On the potential utility of negative relevance feedback in interactive information retrieval , 1996, SIGIR '96.