An Eye-Tracking Study of Query Reformulation

Information about a user's domain knowledge and interest can be important signals for many information retrieval tasks such as query suggestion or result ranking. State-of-the-art user models rely on coarse-grained representations of the user's previous knowledge about a topic or domain. In this paper, we study query refinement using eye-tracking in order to gain precise and detailed insight into which terms the user was exposed to in a search session and which ones they showed a particular interest in. We measure fixations on the term level, allowing for a detailed model of user attention. To allow for a wide-spread exploitation of our findings, we generalize from the restrictive eye-gaze tracking to using more accessible signals: mouse cursor traces. Based on the public API of a popular search engine, we demonstrate how query suggestion candidates can be ranked according to traces of user attention and interest, resulting in significantly better performance than achieved by an attention-oblivious industry solution. Our experiments suggest that modelling term-level user attention can be achieved with great reliability and holds significant potential for supporting a range of traditional IR tasks.

[1]  Thorsten Joachims,et al.  Eye-tracking analysis of user behavior in WWW search , 2004, SIGIR '04.

[2]  Christiane Fellbaum,et al.  Combining Local Context and Wordnet Similarity for Word Sense Identification , 1998 .

[3]  Amanda Spink,et al.  Term relevance feedback and query expansion: relation to design , 1994, SIGIR '94.

[4]  Ted Pedersen,et al.  An Adapted Lesk Algorithm for Word Sense Disambiguation Using WordNet , 2002, CICLing.

[5]  Nicholas J. Belkin,et al.  Ask for Information Retrieval: Part I. Background and Theory , 1997, J. Documentation.

[6]  Samuel Kaski,et al.  Learning to learn implicit queries from gaze patterns , 2008, ICML '08.

[7]  Yang Song,et al.  Optimal rare query suggestion with implicit user feedback , 2010, WWW '10.

[8]  Ryen W. White,et al.  Lessons from the journey: a query log analysis of within-session learning , 2014, WSDM.

[9]  Barbara M. Wildemuth,et al.  The effects of domain knowledge on search tactic formulation , 2004, J. Assoc. Inf. Sci. Technol..

[10]  David Beymer,et al.  WebGazeAnalyzer: a system for capturing and analyzing web reading behavior using eye gaze , 2005, CHI Extended Abstracts.

[11]  Paul-Alexandru Chirita,et al.  Personalized query expansion for the web , 2007, SIGIR.

[12]  Kenneth Ward Church,et al.  Query suggestion using hitting time , 2008, CIKM '08.

[13]  Amanda Spink,et al.  Interaction in Information Retrieval: Selection and Effectiveness of Search Terms , 1997, J. Am. Soc. Inf. Sci..

[14]  Jimmy J. Lin,et al.  Overview of the TREC 2006 ciQA task , 2007, SIGF.

[15]  Ryen W. White,et al.  Characterizing the influence of domain expertise on web search behavior , 2009, WSDM '09.

[16]  Nicholas J. Belkin,et al.  Examining users' knowledge change in the task completion process , 2013, Inf. Process. Manag..

[17]  Douglas W. Oard,et al.  Measuring the Utility of Gaze Detection for Task Modeling: A Preliminary Study , 2006 .

[18]  Kai Kunze,et al.  I know what you are reading: recognition of document types using mobile eye tracking , 2013, ISWC '13.

[19]  Michael R. Lyu,et al.  Learning latent semantic relations from clickthrough data for query suggestion , 2008, CIKM '08.

[20]  John Shawe-Taylor,et al.  Can eyes reveal interest? Implicit queries from gaze patterns , 2009, User Modeling and User-Adapted Interaction.

[21]  Ryen W. White,et al.  Personalizing web search results by reading level , 2011, CIKM '11.

[22]  K. Rayner.  Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[23]  Susan T. Dumais,et al.  Personalizing Search via Automated Analysis of Interests and Activities , 2005, SIGIR.

[24]  Martin Porter,et al.  Snowball: A language for stemming algorithms , 2001 .

[25]  Graeme Hirst,et al.  Lexical chains as representations of context for the detection and correction of malapropisms , 1995 .

[26]  Peter Brusilovsky,et al.  Inferring word relevance from eye-movements of readers , 2011, IUI '11.

[27]  M. Brysbaert,et al.  Age-of-acquisition ratings for 30,000 English words , 2012, Behavior research methods.

[28]  Johanna Enberg,et al.  Query Expansion , 2018, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[29]  M A Just,et al.  A theory of reading: from eye fixations to comprehension. , 1980, Psychological review.

[30]  Arjen P. de Vries,et al.  A combined topical/non-topical approach to identifying web sites for children , 2011, WSDM '11.

[31]  Eugene Agichtein,et al.  Towards predicting web searcher gaze position from mouse movements , 2010, CHI Extended Abstracts.

[32]  Edward Cutrell,et al.  What are you looking for?: an eye-tracking study of information usage in web search , 2007, CHI.

[33]  Robin K. Morris,et al.  Eye movements, word familiarity, and vocabulary acquisition , 2004 .

[34]  Samuel Kaski,et al.  Combining eye movements and collaborative filtering for proactive information retrieval , 2005, SIGIR '05.

[35]  Thorsten Joachims,et al.  Accurately interpreting clickthrough data as implicit feedback , 2005, SIGIR '05.

[36]  O. Mimura.  [Eye movements]. , 1992, Nippon Ganka Gakkai zasshi.

[37]  Christiane Fellbaum,et al.  Lexical Chains as Representations of Context for the Detection and Correction of Malapropisms , 1998 .

[38]  Efthimis N. Efthimiadis,et al.  Interactive query expansion: A user-based evaluation in a relevance feedback environment , 2000, J. Am. Soc. Inf. Sci..

[39]  Martha Palmer,et al.  Verb Semantics and Lexical Selection , 1994, ACL.

[40]  Enhong Chen,et al.  Context-aware query suggestion by mining click-through and session data , 2008, KDD.

[41]  Kai Kunze,et al.  Towards inferring language expertise using eye tracking , 2013, CHI Extended Abstracts.

[42]  Nicholas J. Belkin,et al.  Predicting users' domain knowledge from search behaviors , 2011, SIGIR.

[43]  Samuel Kaski,et al.  Can Relevance be Inferred from Eye Movements in Information Retrieval , 2003 .

[44]  Eric Horvitz,et al.  Patterns of search: analyzing and modeling Web query refinement , 1999 .

[45]  Ryen W. White,et al.  Improving searcher models using mouse cursor activity , 2012, SIGIR '12.

[46]  Andreas Dengel,et al.  Query expansion using gaze-based feedback on the subdocument level , 2008, SIGIR '08.

[47]  Ryen W. White,et al.  No clicks, no problem: using cursor movements to understand and improve search , 2011, CHI.

[48]  Nicholas J. Belkin,et al.  Display time as implicit feedback: understanding task effects , 2004, SIGIR '04.

[49]  Jacek Gwizdka,et al.  Search behaviors in different task types , 2010, JCDL '10.

[50]  Eugene Agichtein,et al.  Beyond dwell time: estimating document relevance from cursor movements and other post-click searcher behavior , 2012, WWW.

[51]  Diane Kelly,et al.  The effects of topic familiarity on information search behavior , 2002, JCDL '02.

[52]  Eugene Agichtein,et al.  Improving search result summaries by using searcher behavior data , 2013, SIGIR.

[53]  Nicholas J. Belkin,et al.  Ask for Information Retrieval: Part II. Results of a Design Study , 1982, J. Documentation.

[54]  Andreas Dengel,et al.  Eye movements as implicit relevance feedback , 2008, CHI Extended Abstracts.

[55]  Martin Chodorow,et al.  Combining local context and wordnet similarity for word sense identification , 1998 .

[56]  Wei Gao,et al.  Cross-lingual query suggestion using query logs of different languages , 2007, SIGIR.

[57]  Karl Gyllstrom,et al.  A comparison of query and term suggestion features for interactive searching , 2009, SIGIR.