Adapting information retrieval systems to user queries

Users enter queries that are short as well as long. The aim of this work is to evaluate techniques that can enable information retrieval (IR) systems to automatically adapt to perform better on such queries. By adaptation we refer to (1) modifications to the queries via user interaction, and (2) detecting that the original query is not a good candidate for modification. We show that the former has the potential to improve mean average precision (MAP) of long and short queries by 40% and 30% respectively, and that simple user interaction can help towards this goal. We observed that after inspecting the options presented to them, users frequently did not select any. We present techniques in this paper to determine beforehand the utility of user interaction to avoid this waste of time and effort. We show that our techniques can provide IR systems with the ability to detect and avoid interaction for unpromising queries without a significant drop in overall performance.

[1]  BuckleyChris,et al.  Using clustering and SuperConcepts within SMART , 2000 .

[2]  W. Bruce Croft,et al.  Ranking robustness: a novel framework to predict query performance , 2006, CIKM '06.

[3]  James Allan,et al.  A Case For Shorter Queries, and Helping Users Create Them , 2007, NAACL.

[4]  Iadh Ounis,et al.  Inferring Query Performance Using Pre-retrieval Predictors , 2004, SPIRE.

[5]  Amanda Spink,et al.  From E-Sex to E-Commerce: Web Search Changes , 2002, Computer.

[6]  C. J. van Rijsbergen,et al.  Information Retrieval , 1979, Encyclopedia of GIS.

[7]  Peter Bruza,et al.  Searching the World Wide Web made easy? the cognitive load imposed by query refinement mechanisms , 1998 .

[8]  Ravi Kumar,et al.  Searching with context , 2006, WWW '06.

[9]  W. Bruce Croft,et al.  Query performance prediction in web search environments , 2007, SIGIR.

[10]  Ellen M. Voorhees,et al.  The TREC 2005 robust track , 2006, SIGF.

[11]  ChengXiang Zhai,et al.  Active feedback in ad hoc information retrieval , 2005, SIGIR '05.

[12]  Peter Ingwersen,et al.  The development of a method for the evaluation of interactive information retrieval systems , 1997, J. Documentation.

[13]  S. Stearns,et al.  Registration of image cubes using multivariate mutual information , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[14]  Clifford Stein,et al.  Introduction to Algorithms, 2nd edition. , 2001 .

[15]  James Allan,et al.  HARD Track Overview in TREC 2004 (Notebook) High Accuracy Retrieval from Documents , 2004 .

[16]  James Allan,et al.  HARD Track Overview in TREC 2003: High Accuracy Retrieval from Documents , 2003, TREC.

[17]  Donna K. Harman,et al.  Overview of the Fifth Text REtrieval Conference (TREC-5) , 1996, TREC.

[18]  James Allan,et al.  INQUERY at TREC-5 , 1996, TREC.

[19]  Ian Ruthven,et al.  Re-examining the potential effectiveness of interactive query expansion , 2003, SIGIR.

[20]  W. Bruce Croft,et al.  I3R: A new approach to the design of document retrieval systems , 1987, J. Am. Soc. Inf. Sci..

[21]  W. Bruce Croft,et al.  I 3 R: a new approach to the design of document retrieval systems , 1987 .

[22]  Thomas H. Cormen,et al.  Introduction to algorithms [2nd ed.] , 2001 .

[23]  Peter G. Anick,et al.  The paraphrase search assistant: terminological feedback for iterative information seeking , 1999, SIGIR '99.

[24]  James Allan,et al.  Selective user interaction , 2007, CIKM '07.

[25]  Stephen E. Robertson,et al.  On GMAP: and other transformations , 2006, CIKM '06.

[26]  Jacob Shapiro,et al.  Constructing Web search queries from the user's information need expressed in a natural language , 2003, SAC '03.

[27]  Mark Magennis,et al.  The potential and actual effectiveness of interactive query expansion , 1997, SIGIR '97.

[28]  W. Bruce Croft,et al.  Query expansion using local and global document analysis , 1996, SIGIR '96.

[29]  Ryen W. White,et al.  Using top-ranking sentences to facilitate effective information access: Book Reviews , 2005 .

[30]  Ragnar Nordlie,et al.  “User revealment”—a comparison of initial queries and ensuing question development in online searching and in human reference interactions , 1999, SIGIR '99.

[31]  W. Bruce Croft,et al.  Predicting query performance , 2002, SIGIR '02.

[32]  Donna K. Harman,et al.  The NRRC reliable information access (RIA) workshop , 2004, SIGIR '04.

[33]  Ronald L. Rivest,et al.  Introduction to Algorithms, Second Edition , 2001 .

[34]  Kenneth Ward Church,et al.  Word Association Norms, Mutual Information, and Lexicography , 1989, ACL.

[35]  Amanda Spink,et al.  Characteristics of question format web queries: an exploratory study , 2002, Inf. Process. Manag..

[36]  Claire Cardie,et al.  Using clustering and SuperConcepts within SMART: TREC 6 , 1997, Inf. Process. Manag..

[37]  Elad Yom-Tov,et al.  What makes a query difficult? , 2006, SIGIR.

[38]  Ryen W. White,et al.  Using top-ranking sentences to facilitate effective information access , 2005, J. Assoc. Inf. Sci. Technol..

[39]  Richard M. Schwartz,et al.  An Algorithm that Learns What's in a Name , 1999, Machine Learning.