Struggling or exploring?: disambiguating long search sessions

Web searchers often exhibit directed search behaviors such as navigating to a particular Website. However, in many circumstances they exhibit different behaviors that involve issuing many queries and visiting many results. In such cases, it is not clear whether the user's rationale is to intentionally explore the results or whether they are struggling to find the information they seek. Being able to disambiguate between these types of long search sessions is important for search engines both in performing retrospective analysis to understand search success, and in developing real-time support to assist searchers. The difficulty of this challenge is amplified since many of the characteristics of exploration (e.g., multiple queries, long duration) are also observed in sessions where people are struggling. In this paper, we analyze struggling and exploring behavior in Web search using log data from a commercial search engine. We first compare and contrast search behaviors along a number dimensions, including query dynamics during the session. We then build classifiers that can accurately distinguish between exploring and struggling sessions using behavioral and topical features. Finally, we show that by considering the struggling/exploring prediction we can more accurately predict search satisfaction.

[1]  Steve Fox,et al.  Evaluating implicit measures to improve web search , 2005, TOIS.

[2]  Jacek Gwizdka,et al.  What Can Searching Behavior Tell Us About the Difficulty of Information Tasks? A Study of Web Navigation , 2007, ASIST.

[3]  Ryen W. White,et al.  Assessing the scenic route: measuring the value of search trails in web logs , 2010, SIGIR.

[4]  Scott B. Huffman,et al.  How well does result relevance predict session satisfaction? , 2007, SIGIR.

[5]  Paul N. Bennett,et al.  Toward whole-session relevance: exploring intrinsic diversity in web search , 2013, SIGIR.

[6]  Ryen W. White,et al.  Exploratory Search: Beyond the Query-Response Paradigm , 2009, Exploratory Search: Beyond the Query-Response Paradigm.

[7]  Nicholas J. Belkin,et al.  Task difficulty and domain knowledge effects on information search behaviors , 2012, ASIST.

[8]  Susan T. Dumais,et al.  Evaluating implicit measures to improve the search experiences , 2003 .

[9]  W. Grove.  Statistical Methods for Rates and Proportions, 2nd ed , 1981 .

[10]  Susan T. Dumais,et al.  Analysis of topic dynamics in web search , 2005, WWW '05.

[11]  Jacek Gwizdka,et al.  Can search systems detect users' task difficulty?: some behavioral signals , 2010, SIGIR '10.

[12]  Doug Downey,et al.  Understanding the relationship between searchers' queries and information goals , 2008, CIKM '08.

[13]  Filip Radlinski,et al.  Query chains: learning to rank from implicit feedback , 2005, KDD '05.

[14]  Yisong Yue,et al.  Beyond position bias: examining result attractiveness as a source of presentation bias in clickthrough data , 2010, WWW '10.

[15]  Dan Morris,et al.  SearchBar: a search-centric web history for task resumption and information re-finding , 2008, CHI.

[16]  Rosie Jones,et al.  Beyond the session timeout: automatic hierarchical segmentation of search topics in query logs , 2008, CIKM '08.

[17]  Ahmed Hassan Awadallah,et al.  A semi-supervised approach to modeling web search satisfaction , 2012, SIGIR '12.

[18]  Ahmed Hassan Awadallah,et al.  Beyond DCG: user behavior as a predictor of a successful search , 2010, WSDM '10.

[19]  Xiaojun Yuan,et al.  Building the trail best traveled: effects of domain knowledge on web search trailblazing , 2012, CHI.

[20]  Martha Palmer,et al.  Verb Semantics and Lexical Selection , 1994, ACL.

[21]  Thorsten Joachims,et al.  Accurately interpreting clickthrough data as implicit feedback , 2005, SIGIR '05.

[22]  Anne Aula,et al.  How does search behavior change as search becomes more difficult? , 2010, CHI.

[23]  James Allan,et al.  Predicting searcher frustration , 2010, SIGIR.

[24]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[25]  M. de Rijke,et al.  A subjunctive exploratory search interface to support media studies researchers , 2012, SIGIR '12.

[26]  Eugene Agichtein,et al.  Find it if you can: a game for modeling different types of web search success using interaction data , 2011, SIGIR.

[27]  Ben Carterette,et al.  Evaluating multi-query sessions , 2011, SIGIR.

[28]  Gary Marchionini,et al.  Exploratory search , 2006, Commun. ACM.

[29]  Charles L. A. Clarke,et al.  The influence of caption features on clickthrough patterns in web search , 2007, SIGIR.

[30]  Ryen W. White,et al.  Exploring the use of labels to shortcut search trails , 2010, SIGIR.

[31]  H. Toutenburg.  Fleiss, J. L.: Statistical Methods for Rates and Proportions. John Wiley & Sons, New York‐London‐Sydney‐Toronto 1973. XIII, 233 S. , 1974 .

[32]  Jeonghyun Kim,et al.  Task difficulty as a predictor and indicator of web searching interaction , 2006, CHI Extended Abstracts.

[33]  Thorsten Joachims,et al.  Evaluating Search Engines using Clickthrough Data , 2002 .

[34]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[35]  Ryen W. White,et al.  Task tours: helping users tackle complex search tasks , 2012, CIKM.

[36]  Ben Shneiderman,et al.  Determining Causes and Severity of End-User Frustration , 2004, Int. J. Hum. Comput. Interact..

[37]  Ryen W. White,et al.  Characterizing and predicting search engine switching behavior , 2009, CIKM.