Modeling Perceived Relevance for Tail Queries without Click-Through Data

Click-through data has been used in various ways in Web search such as estimating relevance between documents and queries. Since only search snippets are perceived by users before issuing any clicks, the relevance induced by clicks are usually called \emph{perceived relevance} which has proven to be quite useful for Web search. While there is plenty of click data for popular queries, very little information is available for unpopular tail ones. These tail queries take a large portion of the search volume but search accuracy for these queries is usually unsatisfactory due to data sparseness such as limited click information. In this paper, we study the problem of modeling perceived relevance for queries without click-through data. Instead of relying on users' click data, we carefully design a set of snippet features and use them to approximately capture the perceived relevance. We study the effectiveness of this set of snippet features in two settings: (1) predicting perceived relevance and (2) enhancing search engine ranking. Experimental results show that our proposed model is effective to predict the relative perceived relevance of Web search results. Furthermore, our proposed snippet features are effective to improve search accuracy for longer tail queries without click-through data.

[1]  Yang Song,et al.  Optimal rare query suggestion with implicit user feedback , 2010, WWW '10.

[2]  Ciya Liao,et al.  A model to estimate intrinsic document relevance from the clickthrough logs of a web search engine , 2010, WSDM '10.

[3]  Tapas Kanungo,et al.  Predicting the readability of short web summaries , 2009, WSDM '09.

[4]  Wei-Ying Ma,et al.  Optimizing web search using web click-through data , 2004, CIKM '04.

[5]  Xiao Li,et al.  Learning query intent from regularized click graphs , 2008, SIGIR '08.

[6]  Daniel E. Rose,et al.  Summary attributes and perceived search quality , 2007, WWW '07.

[7]  Hongyuan Zha,et al.  A General Boosting Method and its Application to Learning Ranking Functions for Web Search , 2007, NIPS.

[8]  Monika Henzinger,et al.  Analysis of a very large web search engine query log , 1999, SIGF.

[9]  Mark Sanderson,et al.  Advantages of query biased summaries in information retrieval , 1998, SIGIR '98.

[10]  Andrei Z. Broder,et al.  Robust classification of rare queries using web knowledge , 2007, SIGIR.

[11]  Yisong Yue,et al.  Beyond position bias: examining result attractiveness as a source of presentation bias in clickthrough data , 2010, WWW '10.

[12]  Thorsten Joachims,et al.  Accurately Interpreting Clickthrough Data as Implicit Feedback , 2017 .

[13]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[14]  Ricardo A. Baeza-Yates,et al.  Extracting semantic relations from query logs , 2007, KDD '07.

[15]  Charles L. A. Clarke,et al.  The influence of caption features on clickthrough patterns in web search , 2007, SIGIR.

[16]  Marcus Fontoura,et al.  Estimating advertisability of tail queries for sponsored search , 2010, SIGIR.

[17]  Nick Craswell,et al.  An experimental comparison of click position-bias models , 2008, WSDM '08.

[18]  James P. Callan,et al.  Passage-level evidence in document retrieval , 1994, SIGIR '94.

[19]  Tie-Yan Liu,et al.  Adapting ranking SVM to document retrieval , 2006, SIGIR.

[20]  Zhaohui Zheng,et al.  Learning to re-rank web search results with multiple pairwise features , 2011, WSDM '11.

[21]  Yuchen Zhang,et al.  Characterizing search intent diversity into click models , 2011, WWW.

[22]  Wei Vivian Zhang,et al.  Comparing Click Logs and Editorial Labels for Training Query Rewriting , 2007 .

[23]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[24]  Doug Downey,et al.  Heads and tails: studies of web search with common and rare queries , 2007, SIGIR.

[25]  Edward Cutrell,et al.  What are you looking for?: an eye-tracking study of information usage in web search , 2007, CHI.

[26]  Benjamin Piwowarski,et al.  A user browsing model to predict search engine click data from past observations. , 2008, SIGIR '08.

[27]  Andrei Z. Broder,et al.  Estimating rates of rare events at multiple resolutions , 2007, KDD '07.

[28]  Matthew Richardson,et al.  Predicting clicks: estimating the click-through rate for new ads , 2007, WWW '07.

[29]  Doug Beeferman,et al.  Agglomerative clustering of a search engine query log , 2000, KDD '00.

[30]  Chao Liu,et al.  Click chain model in web search , 2009, WWW '09.

[31]  Hongyuan Zha,et al.  Global ranking by exploiting user clicks , 2009, SIGIR.

[32]  Gregory N. Hullender,et al.  Learning to rank using gradient descent , 2005, ICML.

[33]  Andrei Z. Broder,et al.  Anatomy of the long tail: ordinary people with extraordinary tastes , 2010, WSDM '10.

[34]  J. Avery,et al.  The long tail. , 1995, Journal of the Tennessee Medical Association.

[35]  Olivier Chapelle,et al.  A dynamic bayesian network click model for web search ranking , 2009, WWW '09.

[36]  Mehran Sahami,et al.  A web-based kernel function for measuring the similarity of short text snippets , 2006, WWW '06.

[37]  Susan T. Dumais,et al.  Learning user interaction models for predicting web search result preferences , 2006, SIGIR.