An Extended Query Performance Prediction Framework Utilizing Passage-Level Information

We show that document-level post-retrieval query performance prediction (QPP) methods are mostly suited for short query prediction tasks; such methods perform significantly worse in verbose (long and informative) query prediction settings. To address the prediction quality gap among query lengths, we propose a novel passage-level post-retrieval QPP framework. Our empirical analysis demonstrates that, those QPP methods that utilize passage-level information are much better suited for verbose QPP settings. Moreover, our proposed predictors, which utilize both document-level and passage-level information provide a more robust prediction which is less sensitive to query length.

[1]  Oren Kurland,et al.  Utilizing Passage-Based Language Models for Document Retrieval , 2008, ECIR.

[2]  Jimmy J. Lin,et al.  Quantitative evaluation of passage retrieval algorithms for question answering , 2003, SIGIR.

[3]  Oren Kurland,et al.  Back to the roots: a probabilistic framework for query-performance prediction , 2012, CIKM.

[4]  W. Bruce Croft,et al.  Predicting query performance , 2002, SIGIR '02.

[5]  Oren Kurland,et al.  Clarity re-visited , 2012, SIGIR '12.

[6]  W. Bruce Croft,et al.  Modeling higher-order term dependencies in information retrieval using query hypergraphs , 2012, SIGIR '12.

[7]  W. Bruce Croft,et al.  Passage retrieval based on language models , 2002, CIKM '02.

[8]  W. Bruce Croft,et al.  Relevance-Based Language Models , 2001, SIGIR '01.

[9]  John D. Lafferty,et al.  A study of smoothing methods for language models applied to Ad Hoc information retrieval , 2001, SIGIR '01.

[10]  Oren Kurland,et al.  Predicting Query Performance by Query-Drift Estimation , 2009, TOIS.

[11]  James P. Callan,et al.  Passage-level evidence in document retrieval , 1994, SIGIR '94.

[12]  Haggai Roitman.  An Enhanced Approach to Query Performance Prediction Using Reference Lists , 2017, SIGIR.

[13]  Stephen E. Robertson,et al.  Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval , 1994, SIGIR '94.

[14]  Haggai Roitman.  Query Performance Prediction using Passage Information , 2018, SIGIR.

[15]  Ingemar J. Cox,et al.  On ranking the effectiveness of searches , 2006, SIGIR.

[16]  W. Bruce Croft,et al.  Query performance prediction in web search environments , 2007, SIGIR.

[17]  Haggai Roitman,et al.  Robust Standard Deviation Estimation for Query Performance Prediction , 2017, ICTIR.

[18]  W. Bruce Croft,et al.  A Language Modeling Approach to Information Retrieval , 1998, SIGIR Forum.

[19]  Djoerd Hiemstra,et al.  A survey of pre-retrieval query performance predictors , 2008, CIKM '08.

[20]  Oren Kurland,et al.  Query performance prediction for IR , 2012, SIGIR '12.

[21]  Mathias Géry,et al.  BM25t: a BM25 extension for focused information retrieval , 2012, Knowledge and Information Systems.

[22]  Haggai Roitman,et al.  Enhanced Mean Retrieval Score Estimation for Query Performance Prediction , 2017, ICTIR.

[23]  Ido Guy,et al.  Searching by Talking: Analysis of Voice Queries on Mobile Web Search , 2016, SIGIR.

[24]  W. Bruce Croft,et al.  Precision prediction based on ranked list coherence , 2006, Information Retrieval.

[25]  David Konopnicki,et al.  Unsupervised Query-Focused Multi-Document Summarization using the Cross Entropy Method , 2017, SIGIR.

[26]  Oren Kurland,et al.  Predicting the performance of passage retrieval for question answering , 2012, CIKM.

[27]  Elad Yom-Tov,et al.  What makes a query difficult? , 2006, SIGIR.

[28]  Oren Kurland,et al.  On identifying representative relevant documents , 2010, CIKM.

[29]  Manish Gupta,et al.  Information Retrieval with Verbose Queries , 2015, Found. Trends Inf. Retr..