ICTIR Tutorial: Modern Query Performance Prediction: Theory and Practice

Query performance prediction (QPP) is a core information retrieval (IR) task whose primary goal is to assess retrieval quality in the absence of relevance judgments. Applications of QPP are numerous, and include, among others, automatic query reformulation, fusion and ranker selection, distributed search and content analysis. The main objective of this tutorial is to introduce recent advances in the sub-research area of QPP in IR, covering both theory and applications. On the theoretical side, we will introduce modern QPP frameworks, which have advanced our understanding of the core QPP task. On the application side, the tutorial will set the connection between QPP theory and its usage in various modern IR applications, discussing the pros and cons, limitations, challenges and open research questions.

[1]  Elad Yom-Tov,et al.  What makes a query difficult? , 2006, SIGIR.

[2]  Anderson Rocha,et al.  Relevance prediction in similarity-search systems using extreme value theory , 2019, J. Vis. Commun. Image Represent..

[3]  Oren Kurland,et al.  Using the cross-entropy method to re-rank search results , 2014, SIGIR.

[4]  Oren Kurland,et al.  Estimating query representativeness for query-performance prediction , 2013, SIGIR.

[5]  Jelena Jovanovic,et al.  Neural Embedding-Based Metrics for Pre-retrieval Query Performance Prediction , 2020, ECIR.

[6]  Oren Kurland,et al.  Query Performance Prediction Using Reference Lists , 2016, ACM Trans. Inf. Syst..

[7]  Oren Kurland,et al.  Query-Performance Prediction Using Minimal Relevance Feedback , 2013, ICTIR.

[8]  J. Shane Culpepper,et al.  Dynamic Shard Cutoff Prediction for Selective Search , 2018, SIGIR.

[9]  Oren Kurland,et al.  Query Performance Prediction for Pseudo-Feedback-Based Retrieval , 2019, SIGIR.

[10]  J. Shane Culpepper,et al.  Neural Query Performance Prediction using Weak Supervision from Multiple Signals , 2018, SIGIR.

[11]  Ronan Cummins,et al.  Document Score Distribution Models for Query Performance Inference and Prediction , 2014, TOIS.

[12]  W. Bruce Croft,et al.  Query performance prediction in web search environments , 2007, SIGIR.

[13]  J. Shane Culpepper,et al.  Information Needs, Queries, and Query Performance Prediction , 2019, SIGIR.

[14]  Haggai Roitman,et al.  Enhanced Mean Retrieval Score Estimation for Query Performance Prediction , 2017, ICTIR.

[15]  Djoerd Hiemstra,et al.  Query Performance Prediction: Evaluation Contrasted with Effectiveness , 2010, ECIR.

[16]  Oren Kurland,et al.  Back to the roots: a probabilistic framework for query-performance prediction , 2012, CIKM.

[17]  Elad Yom-Tov,et al.  Estimating the query difficulty for information retrieval , 2010, Synthesis Lectures on Information Concepts, Retrieval, and Services.

[18]  Haggai Roitman,et al.  Enhanced Performance Prediction of Fusion-based Retrieval , 2018, ICTIR.

[19]  W. Bruce Croft,et al.  Predicting query performance , 2002, SIGIR '02.

[20]  W. Bruce Croft,et al.  Performance Prediction for Non-Factoid Question Answering , 2019, ICTIR.

[21]  Haggai Roitman.  An Enhanced Approach to Query Performance Prediction Using Reference Lists , 2017, SIGIR.

[22]  Oren Kurland,et al.  Query-performance prediction and cluster ranking: two sides of the same coin , 2012, CIKM.

[23]  Haggai Roitman,et al.  An Extended Query Performance Prediction Framework Utilizing Passage-Level Information , 2018, ICTIR.

[24]  Oren Kurland,et al.  Predicting the performance of passage retrieval for question answering , 2012, CIKM.

[25]  Oren Kurland,et al.  Predicting Query Performance by Query-Drift Estimation , 2009, TOIS.

[26]  Guy Feigenblat,et al.  Query Performance Prediction for Multifield Document Retrieval , 2020, ICTIR.

[27]  Haggai Roitman,et al.  Robust Standard Deviation Estimation for Query Performance Prediction , 2017, ICTIR.

[28]  Oren Kurland,et al.  Query-performance prediction: setting the expectations straight , 2014, SIGIR.

[29]  Oren Kurland,et al.  Predicting query performance for fusion-based retrieval , 2012, CIKM.

[30]  Oren Kurland,et al.  Query performance prediction for entity retrieval , 2014, SIGIR.

[31]  Djoerd Hiemstra,et al.  A survey of pre-retrieval query performance predictors , 2008, CIKM '08.

[32]  Oren Kurland,et al.  Using statistical decision theory and relevance models for query-performance prediction , 2010, SIGIR.

[33]  Gareth J. F. Jones,et al.  Estimating Gaussian mixture models in the local neighbourhood of embedded word vectors for query performance prediction , 2019, Inf. Process. Manag..

[34]  Oren Kurland,et al.  A Unified Framework for Post-Retrieval Query-Performance Prediction , 2011, ICTIR.

[35]  Lior Rokach,et al.  Wikipedia-based query performance prediction , 2014, SIGIR.

[36]  Haggai Roitman,et al.  Normalized Query Commitment Revisited , 2019, SIGIR.

[37]  Haggai Roitman,et al.  As Stable As You Are: Re-ranking Search Results using Query-Drift Analysis , 2018, HT.

[38]  Samuel B. Williams,et al.  ASSOCIATION FOR COMPUTING MACHINERY , 2000 .

[39]  Ismail Sengör Altingövde,et al.  Query Performance Prediction for Aspect Weighting in Search Result Diversification , 2014, CIKM.

[40]  Guy Feigenblat,et al.  A Study of Query Performance Prediction for Answer Quality Determination , 2019, ICTIR.