Only forward?: toward understanding human visual behaviour when examining search results

Search is near-ubiquitous in human society, being used for entertainment, health, financial and business information seeking. Traditional methods of search evaluation have assumed that searchers move forward through search results in a linear manner; early eye tracking studies have suggested the same. Recent research, though, including eye-tracking data, has demonstrated a number of counter-cases, particularly for search in complex conditions. These examples highlight how little we understand where humans look or what they are doing when examining search results: in this paper we undertake a broad survey of the literature to refine this research question, and suggest avenues for developing a model of behaviour.

[1]  Thorsten Joachims,et al.  Eye-tracking analysis of user behavior in WWW search , 2004, SIGIR '04.

[2]  Timothy Baldwin,et al.  Understanding User Behavior in Job and Talent Search: An Initial Investigation , 2017, eCOM@SIGIR.

[3]  Ryen W. White,et al.  Large-scale analysis of individual and task differences in search result page examination strategies , 2012, WSDM '12.

[4]  Thorsten Joachims,et al.  Accurately interpreting clickthrough data as implicit feedback , 2005, SIGIR '05.

[5]  Annika Hinze,et al.  Book selection behavior in the physical library: implications for ebook collections , 2012, JCDL '12.

[6]  Jakob Nielsen,et al.  Eyetracking Web Usability , 2009 .

[7]  George Buchanan,et al.  Boxing clever: how searchers use and adapt to a one-box library search , 2013, OZCHI.

[8]  Andrew Turpin,et al.  Why batch and user evaluations do not give the same results , 2001, SIGIR '01.

[9]  Nick Bryan-Kinns,et al.  Use of multiple digital libraries: a case study , 2001, JCDL '01.

[10]  Alistair Moffat,et al.  What Users Do: The Eyes Have It , 2013, AIRS.

[11]  Jacek Gwizdka,et al.  Search behaviors in different task types , 2010, JCDL '10.

[12]  Falk Scholer,et al.  Examining the Pseudo-Standard Web Search Engine Results Page , 2006, Aust. J. Intell. Inf. Process. Syst..

[13]  Mark Sanderson,et al.  Test Collection Based Evaluation of Information Retrieval Systems , 2010, Found. Trends Inf. Retr..

[14]  Dana McKay,et al.  A jump to the left (and then a step to the right): reading practices within academic ebooks , 2011, OZCHI.

[15]  Bing Pan,et al.  The determinants of web page viewing behavior: an eye-tracking study , 2004, ETRA.

[16]  Peter Bailey,et al.  User Variability and IR System Evaluation , 2015, SIGIR.

[17]  Marcia J. Bates,et al.  A Profile of End-User Searching Behavior by Humanities Scholars: The Getty Online Searching Project Report No. 2 , 1993, J. Am. Soc. Inf. Sci..

[18]  Charles L. A. Clarke,et al.  The influence of caption features on clickthrough patterns in web search , 2007, SIGIR.

[19]  Marcia J. Bates,et al.  A profile of end‐user searching behavior by humanities scholars: The Getty Online Searching Project Report No. 2 , 1993 .

[20]  A. Blandford,et al.  Analytical usability evaluation for digital libraries: a case study , 2004, Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries, 2004..

[21]  Ann Blandford,et al.  Prioritisation, Resources and Search Terms: A Study of Decision-Making at the Virtual Reference Desk , 2008, ECDL.

[22]  Elke Greifeneder,et al.  The Impact of Distraction in Natural Environments on User Experience Research , 2011, TPDL.

[23]  Andreas Dengel,et al.  Attentive documents: Eye tracking as implicit feedback for information retrieval and beyond , 2012, TIIS.

[24]  Michael Khoo,et al.  What Would 'Google' Do? Users' Mental Models of a Digital Library Search Engine , 2012, TPDL.

[25]  Charles L. A. Clarke,et al.  A comparative analysis of cascade measures for novelty and diversity , 2011, WSDM '11.

[26]  Diane Kelly,et al.  Methods for Evaluating Interactive Information Retrieval Systems with Users , 2009, Found. Trends Inf. Retr..

[27]  Jaime Arguello,et al.  Development and Evaluation of Search Tasks for IIR Experiments using a Cognitive Complexity Framework , 2015, ICTIR.

[28]  Alistair Moffat,et al.  Rank-biased precision for measurement of retrieval effectiveness , 2008, TOIS.

[29]  Pertti Vakkari,et al.  Result List Actions in Fiction Search , 2015, JCDL.

[30]  Meng Wang,et al.  Investigating Examination Behavior of Image Search Users , 2017, SIGIR.

[31]  Andrei Broder,et al.  A taxonomy of web search , 2002, SIGF.

[32]  George Buchanan,et al.  Investigating Document Triage on Paper and Electronic Media , 2007, ECDL.

[33]  George Buchanan,et al.  Phonological working memory impacts on information searching: an investigation of dyslexia , 2012, IIiX.

[34]  Edward Cutrell,et al.  An eye tracking study of the effect of target rank on web search , 2007, CHI.

[35]  George Buchanan,et al.  Cognitive economy and satisficing in information seeking: A longitudinal study of undergraduate information behavior , 2009, J. Assoc. Inf. Sci. Technol..

[36]  Ryen W. White,et al.  Characterizing the influence of domain expertise on web search behavior , 2009, WSDM '09.

[37]  Catherine C. Marshall,et al.  Turning the page on navigation , 2005, Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '05).

[38]  Amanda Spink,et al.  Searching the Web: the public and their queries , 2001 .

[39]  Tony Stockman,et al.  A comparative analysis of the information-seeking behavior of visually impaired and sighted searchers , 2012, J. Assoc. Inf. Sci. Technol..

[40]  Ann Blandford,et al.  Investigating the information-seeking behaviour of academic lawyers: From Ellis's model to design , 2008, Inf. Process. Manag..

[41]  George Buchanan,et al.  Information Seeking by Humanities Scholars , 2005, ECDL.

[42]  Daqing He,et al.  Searching, browsing, and clicking in a search session: changes in user behavior by task and over time , 2014, SIGIR.

[43]  Edward Cutrell,et al.  What are you looking for?: an eye-tracking study of information usage in web search , 2007, CHI.

[44]  Falk Scholer,et al.  User performance versus precision measures for simple search tasks , 2006, SIGIR.

[45]  Gary Marchionini,et al.  Information Seeking in Electronic Environments , 1995 .