A Study of Query Performance Prediction for Answer Quality Determination

We study a constrained retrieval setting in which either a single qualitative answer is provided as a response to a user-query or none. Given a user-query and the "best" answer that was retrieved from the underlying search engine, we wish to determine whether or not to accept it. To address this challenge, we propose an answer quality determination approach which leverages a novel set of answer-level query performance prediction (QPP) features, derived from a couple of recent discriminative QPP frameworks. Using various search benchmarks with both ad-hoc retrieval and non-factoid question answering (QA) tasks, we demonstrate the effectiveness of our approach.

[1]  Tie-Yan Liu,et al.  Learning to rank for information retrieval , 2009, SIGIR.

[2]  Bowen Zhou,et al.  Applying deep learning to answer selection: A study and an open task , 2015, 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU).

[3]  W. Bruce Croft,et al.  A framework to predict the quality of answers with non-textual features , 2006, SIGIR.

[4]  Oren Kurland,et al.  On identifying representative relevant documents , 2010, CIKM.

[5]  Oren Kurland,et al.  Query-performance prediction: setting the expectations straight , 2014, SIGIR.

[6]  Xu Chen,et al.  Towards Conversational Search and Recommendation: System Ask, User Respond , 2018, CIKM.

[7]  Mihai Surdeanu,et al.  Learning to Rank Answers to Non-Factoid Questions from Web Collections , 2011, CL.

[8]  Imed Zitouni,et al.  Predicting User Satisfaction with Intelligent Assistants , 2016, SIGIR.

[9]  Djoerd Hiemstra,et al.  A survey of pre-retrieval query performance predictors , 2008, CIKM '08.

[10]  Doron Cohen,et al.  From "More Like This" to "Better Than This" , 2016, ICTIR.

[11]  Moshe Tennenholtz,et al.  The search duel: a response to a strong ranker , 2014, SIGIR.

[12]  Haggai Roitman,et al.  An Extended Query Performance Prediction Framework Utilizing Passage-Level Information , 2018, ICTIR.

[13]  Nobuhiro Kaji,et al.  Prediction of Prospective User Engagement with Intelligent Assistants , 2016, ACL.

[14]  Ismail Sengör Altingövde,et al.  Query Performance Prediction for Aspect Weighting in Search Result Diversification , 2014, CIKM.

[15]  Oren Kurland,et al.  Query performance prediction for IR , 2012, SIGIR '12.

[16]  Craig MacDonald,et al.  On the usefulness of query features for learning to rank , 2012, CIKM.

[17]  Mark Sanderson,et al.  Towards Understanding the Impact of Length in Web Search Result Summaries over a Speech-only Communication Channel , 2015, SIGIR.

[18]  Haggai Roitman,et al.  Enhanced Mean Retrieval Score Estimation for Query Performance Prediction , 2017, ICTIR.

[19]  W. Bruce Croft,et al.  WikiPassageQA: A Benchmark Collection for Research on Non-factoid Answer Passage Retrieval , 2018, SIGIR.

[20]  Shengli Wu,et al.  Query Performance Prediction By Considering Score Magnitude and Variance Together , 2014, CIKM.