A Study of Immediate Requery Behavior in Search

When search results fail to satisfy users» information needs, users often reformulate their search query in the hopes of receiving better results. In many cases, users immediately requery without clicking on any search results. In this paper, we report on a user study designed to investigate the rate at which users immediately reformulate at different levels of search quality. We had users search for answers to questions as we manipulated the placement of the only relevant document in a ranked list of search results. We show that as the quality of search results decreases, the probability of immediately requerying increases. We find that users can quickly decide to immediately reformulate, and the time to immediately reformulate appears to be independent of the quality of the search results. Finally, we show that there appears to be two types of users. One group has a high probability of immediately reformulating and the other is unlikely to immediately reformulate unless no relevant documents can be found in the search results. While requerying takes time, it is the group of users who are more likely to immediately requery that are able to able find answers to questions the fastest.

[1]  Carol Peters,et al.  Report on the SIGIR 2009 workshop on the future of IR evaluation , 2009, SIGF.

[2]  Anthony Jameson,et al.  Depth- and breadth-first processing of search result lists , 2004, CHI EA '04.

[3]  Mark D. Smucker,et al.  Report on the SIGIR 2010 workshop on the simulation of interaction , 2011, SIGF.

[4]  Gabriella Kazai,et al.  Tolerance to irrelevance: a user-effort oriented evaluation of retrieval systems without predefined retrieval unit , 2004 .

[5]  Sofia Stamou,et al.  Queries without Clicks: Successful or Failed Searches? , 2009 .

[6]  Jane Li,et al.  Good abandonment in mobile and PC internet search , 2009, SIGIR.

[7]  Ryen W. White,et al.  Leaving so soon?: understanding and predicting web search abandonment rationales , 2012, CIKM.

[8]  Charles L. A. Clarke,et al.  Report on the SIGIR 2013 workshop on modeling user behavior for information retrieval evaluation (MUBE 2013) , 2013, SIGIR Forum.

[9]  Sofia Stamou,et al.  Interpreting User Inactivity on Search Results , 2010, ECIR.

[10]  Ben Carterette,et al.  Simulating simple user behavior for system effectiveness evaluation , 2011, CIKM '11.

[11]  Edward Cutrell,et al.  What are you looking for?: an eye-tracking study of information usage in web search , 2007, CHI.

[12]  Charles L. A. Clarke,et al.  Modeling Optimal Switching Behavior , 2016, CHIIR.

[13]  Filip Radlinski,et al.  How does clickthrough data reflect retrieval quality? , 2008, CIKM '08.

[14]  Filip Radlinski,et al.  Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search , 2007, TOIS.

[15]  Ryen W. White,et al.  No clicks, no problem: using cursor movements to understand and improve search , 2011, CHI.

[16]  Peter Bailey,et al.  Modeling decision points in user search behavior , 2014, IIiX.

[17]  Diane Kelly,et al.  Using information scent and need for cognition to understand online search behavior , 2014, SIGIR.

[18]  David Maxwell,et al.  Validating simulated interaction for retrieval evaluation , 2017, Information Retrieval Journal.

[19]  Charles L. A. Clarke,et al.  Time-based calibration of effectiveness measures , 2012, SIGIR '12.

[20]  Päivi Majaranta,et al.  Eye-Tracking Reveals the Personal Styles for Search Result Evaluation , 2005, INTERACT.

[21]  Susan T. Dumais,et al.  Individual differences in gaze patterns for web search , 2010, IIiX.

[22]  Filip Radlinski,et al.  Search Engines that Learn from Implicit Feedback , 2007, Computer.

[23]  Kalervo Järvelin,et al.  Time drives interaction: simulating sessions in diverse searching environments , 2012, SIGIR '12.

[24]  Thorsten Joachims,et al.  Accurately interpreting clickthrough data as implicit feedback , 2005, SIGIR '05.

[25]  Ling Xia,et al.  Eye tracking and online search: Lessons learned and challenges ahead , 2008, J. Assoc. Inf. Sci. Technol..

[26]  Thorsten Joachims,et al.  Eye-tracking analysis of user behavior in WWW search , 2004, SIGIR '04.

[27]  Leif Azzopardi,et al.  The economics in interactive information retrieval , 2011, SIGIR.

[28]  David Maxwell,et al.  Searching and Stopping: An Analysis of Stopping Rules and Strategies , 2015, CIKM.

[29]  Charles L. A. Clarke,et al.  Modeling user variance in time-biased gain , 2012, HCIR '12.

[30]  Areej Al-Wabil,et al.  Visual Analysis of Dyslexia on Search , 2017, CHIIR.

[31]  Diane Kelly,et al.  Online search stopping behaviors: An investigation of query abandonment and task stopping , 2014, ASIST.

[32]  Marti A. Hearst.  Search User Interfaces , 2009 .