Towards Inferring Web Page Relevance — An Eye-Tracking Study

We present initial results from a project, in which we examined feasibility of inferring web page relevance from eye-tracking data. We conduced a controlled, lab-based Web search experiment, in which participants conducted assigned information search tasks on Wikipedia. We performed analyses of variance as well as employed classification algorithms in order to predict user perceived Web page relevance. Our findings demonstrate that it is feasible to infer document relevance from eye-tracking data on Web pages. The results indicate that eye fixation duration, pupil size and the probability of continuing reading are good predictors of Web page relevance. Our work extends results from previous studies of text document search that were conducted under more constrained human-information interaction.

[1]  Filip Radlinski,et al.  Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search , 2007, TOIS.

[2]  Fang Chen,et al.  Pupillary Response Based Cognitive Workload Measurement under Luminance Changes , 2011, INTERACT.

[3]  Pia Borlund,et al.  The concept of relevance in IR , 2003, J. Assoc. Inf. Sci. Technol..

[4]  Dagobert Soergel,et al.  Relevance: An improved framework for explicating the notion , 2013, J. Assoc. Inf. Sci. Technol..

[5]  Edward Cutrell,et al.  An eye tracking study of the effect of target rank on web search , 2007, CHI.

[6]  Bing Pan,et al.  The determinants of web page viewing behavior: an eye-tracking study , 2004, ETRA.

[7]  Jacek Gwizdka,et al.  Task and user effects on reading patterns in information search , 2011, Interact. Comput..

[8]  Jacek Gwizdka,et al.  Characterizing relevance with eye-tracking measures , 2014, IIiX.

[9]  Tefko Saracevic Relevance: A review of the literature and a framework for thinking on the notion in information science. Part III: Behavior and effects of relevance , 2007 .

[10]  Andrew Howes,et al.  Strategies for Guiding Interactive Search: An Empirical Investigation Into the Consequences of Label Relevance for Assessment and Selection , 2008, Hum. Comput. Interact..

[11]  Daniel M. Russell,et al.  Discriminating the relevance of web search results with measures of pupil size , 2009, CHI.

[12]  Jacek Gwizdka,et al.  YASFIIRE: yet another system for IIR evaluation , 2014, IIiX.

[13]  Thorsten Joachims,et al.  Eye-tracking analysis of user behavior in WWW search , 2004, SIGIR '04.

[14]  Luca Mainardi,et al.  Characterization of affective states by pupillary dynamics and autonomic correlates , 2013, Front. Neuroeng..

[15]  Peter Ingwersen,et al.  Dimensions of relevance , 2000, Inf. Process. Manag..