Passage feedback with IRIS

We compare a user-defined passage feedback (pf) system to a document feedback (df) system. Df employed the adaptive linear model for retrieval, while pf used weighted query expansion based on positive and negative feedback. Twenty-four searchers performed the same six tasks in varying search and system-order per TREC-8 guidelines. We hypothesized that pf, which featured interactive query expansion, would outperform df, which relied on automatic query expansion. Initial analysis appeared to reject this hypothesis, as df showed slightly higher overall performance than pf. However, analysis by system-order groups indicates only the first pf use had lower performance. These data suggest that pf was more difficult to learn than df, though the second pf use yielded competitive performance. If performance of pf is indeed affected by learning, an improved pf system with usability enhancements may prove to be an effective mechanism for interactive information retrieval.

[1]  Amit Singhal,et al.  Pivoted document length normalization , 1996, SIGIR 1996.

[2]  Chris Buckley,et al.  Using Query Zoning and Correlation Within SMART: TREC 5 , 1996, TREC.

[3]  Nicholas J. Belkin,et al.  Braque: Design of an Interface to Support User Interaction in Information Retrieval , 1993, Inf. Process. Manag..

[4]  Ayse Goker.  Context learning in Okapi , 1997 .

[5]  Ross Wilkinson,et al.  Effective retrieval of structured documents , 1994, SIGIR '94.

[6]  Mike Gatford,et al.  Peeling the onion: Okapi system architecture and software design issues , 1997, J. Documentation.

[7]  Venkata Subramaniam,et al.  Information Retrieval: Data Structures & Algorithms , 1992 .

[8]  Hans Peter Luhn,et al.  A Statistical Approach to Mechanized Encoding and Searching of Literary Information , 1957, IBM J. Res. Dev..

[9]  Gerard Salton,et al.  Improving retrieval performance by relevance feedback , 1997, J. Am. Soc. Inf. Sci..

[10]  Justin Zobel,et al.  Passage retrieval revisited , 1997, SIGIR '97.

[11]  Nicholas J. Belkin,et al.  Rutgers' TREC-6 Interactive Track Experience , 1997, TREC.

[12]  Yiyu Yao,et al.  Evaluation of an adaptive linear model , 1991 .

[13]  Nicholas J. Belkin,et al.  Research on information interaction and intelligent information provision mechanisms , 1986, J. Inf. Sci..

[14]  Nicholas J. Belkin,et al.  Cases, scripts, and information-seeking strategies: On the design of interactive information retrieval systems , 1995 .

[15]  Stephen E. Robertson,et al.  Laboratory experiments with Okapi: participation in the TREC programme , 1997, J. Documentation.

[16]  Chris Buckley,et al.  New Retrieval Approaches Using SMART: TREC 4 , 1995, TREC.

[17]  Steve Walker,et al.  Transaction logging , 1997, J. Documentation.

[18]  William M. Shaw,et al.  Interactive Retrieval using IRIS: TREC-6 Experiments , 1997, TREC.

[19]  Lokman I. Meho,et al.  IRIS at TREC-7 , 1998, TREC.

[20]  Micheline Hancock-Beaulieu,et al.  Interactive Okapi at TREC-6 , 1997, TREC.

[21]  Ricardo Baeza-Yates,et al.  Information Retrieval: Data Structures and Algorithms , 1992 .

[22]  Brenda Dervin,et al.  Given a context by any other name: methodological tools for taming the unruly beast , 1997 .

[23]  Ellen M. Voorhees,et al.  The fifth text REtrieval conference (TREC-5) , 1997 .

[24]  Ellen M. Voorhees,et al.  The Eighth Text REtrieval Conference (TREC-8) , 2000 .

[25]  M. F. Porter,et al.  An algorithm for suffix stripping , 1997 .

[26]  James Allan,et al.  Automatic Query Expansion Using SMART: TREC 3 , 1994, TREC.

[27]  Donna Harman,et al.  Information Processing and Management , 2022 .

[28]  Nicholas J. Belkin,et al.  Rutgers' TREC 2001 Interactive Track Experience , 2001, TREC.

[29]  Peter C. Fishburn,et al.  Utility theory for decision making , 1970 .

[30]  Yiyu Yao,et al.  Query formulation in linear retrieval models , 1990, J. Am. Soc. Inf. Sci..

[31]  Robert Krovetz,et al.  Viewing morphology as an inference process , 1993, Artif. Intell..

[32]  Nicholas J. Belkin,et al.  Ask for Information Retrieval: Part I. Background and Theory , 1997, J. Documentation.

[33]  Micheline Beaulieu,et al.  Experiments on interfaces to support query expansion , 1997, J. Documentation.

[34]  Peter Ingwersen,et al.  Cognitive Perspectives of Information Retrieval Interaction: Elements of a Cognitive IR Theory , 1996, J. Documentation.

[35]  James Allan,et al.  Approaches to passage retrieval in full text information systems , 1993, SIGIR.

[36]  Stephen E. Robertson,et al.  Evaluating Interactive Systems in TREC , 1996, J. Am. Soc. Inf. Sci..

[37]  Colleen Cool,et al.  Recognition of stages in the user's information‐seeking process during online searching by novice searchers , 1992 .

[38]  Alistair Moffat,et al.  Efficient Retrieval of Partial Documents , 1995, Inf. Process. Manag..

[39]  S. K. Michael Wong,et al.  Linear structure in information retrieval , 1988, SIGIR '88.

[40]  Charles T. Meadow,et al.  A Computer Intermediary for Interactive Database Searching. I. Design , 2007, J. Am. Soc. Inf. Sci..

[41]  Kelly Maglaughlin,et al.  IRIS at TREC-8 , 1999, TREC.

[42]  Tefko Saracevic,et al.  Modeling Interaction in Information Retrieval (IR): A Review and Proposal. , 1996 .