An Introduction to Prognostic Search

Implicit relevance feedback has received wide attention recently, as a means to capture the search context in improving search accuracy. However, implicit feedback is usually not available to public or even research communities at large for reasons like being a potential threat to privacy of web users. This makes it difficult to experiment and evaluate web search related research and especially web search personalization algorithms. Given these problems, we are motivated towards an artificial way of creating user relevance feedback, based on insights from query log analysis. We call this simulated feedback. We believe that simulated feedback can be immensely beneficial to web search engine and personalization research communities by greatly reducing efforts involved in collecting user feedback. The benefits from “Simulated feedback” are—It is easy to obtain and also the process of obtaining the feedback data is repeatable and customizable. In this chapter, we describe a simple yet effective approach for creating simulated feedback. We evaluated our system using clickthrough data of a set of real world users and achieved 65% accuracy in generating click-through data of those users.

[1]  Nicholas J. Belkin,et al.  Reading time, scrolling and interaction: exploring implicit sources of user preferences for relevance feedback , 2001, Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.

[2]  Feng Qiu,et al.  Automatic identification of user interest for personalized search , 2006, WWW '06.

[3]  ChengXiang Zhai,et al.  Implicit user modeling for personalized search , 2005, CIKM '05.

[4]  Thorsten Joachims,et al.  Accurately interpreting clickthrough data as implicit feedback , 2005, SIGIR '05.

[5]  Susan T. Dumais,et al.  Learning user interaction models for predicting web search result preferences , 2006, SIGIR.

[6]  Steve Fox,et al.  Evaluating implicit measures to improve web search , 2005, TOIS.

[7]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[8]  Mark Claypool,et al.  Implicit interest indicators , 2001, IUI '01.

[9]  Danielle S. McNamara,et al.  Handbook of latent semantic analysis , 2007 .

[10]  Ryen W. White,et al.  Evaluating implicit feedback models using searcher simulations , 2005, TOIS.

[11]  Masatoshi Yoshikawa,et al.  Adaptive web search based on user profile constructed without any effort from users , 2004, WWW '04.

[12]  Gerard Salton,et al.  The SMART Retrieval System—Experiments in Automatic Document Processing , 1971 .

[13]  Thorsten Joachims,et al.  Eye-tracking analysis of user behavior in WWW search , 2004, SIGIR '04.

[14]  Longbing Cao,et al.  In-depth behavior understanding and use: The behavior informatics approach , 2010, Inf. Sci..

[15]  Filip Radlinski,et al.  Evaluating the Robustness of Learning from Implicit Feedback , 2006, ArXiv.