Investigating Examination Behavior in Mobile Search

Examination is one of the most important user interactions in Web search. A number of works studied examination behavior in Web search and helped researchers better understand how users allocate their attention on search engine result pages (SERPs). Compared to desktop search, mobile search has a number of differences such as fewer results on the screen. These differences bring in mobile-specific factors affecting users' examination behavior. However, there still lacks research on users' attention allocation mechanism via viewports in mobile search. Therefore, we design a lab-based study to collect user's rich interaction behavior in mobile search. Based on the collected data, we first analyze how users examine SERPs and allocate their attention to heterogeneous results. Then we investigate the effect of mobile-specific factors and other common factors on users allocating attention. Finally, we apply the findings of user attention allocation from the user study into click model construction efforts, which significantly improves the state-of-the-art click model. Our work brings insights into a better understanding of users' interaction patterns in mobile search and may benefit other mobile search-related research.

[1]  Fernando Diaz,et al.  Robust models of mouse movement on dynamic web search results pages , 2013, CIKM.

[2]  Benjamin Piwowarski,et al.  A user browsing model to predict search engine click data from past observations. , 2008, SIGIR '08.

[3]  Andrew T. Duchowski,et al.  Using Eye Tracking to Evaluate Alternative Search Results Interfaces , 2005 .

[4]  Eugene Agichtein,et al.  Mining touch interaction data on mobile devices to predict web search result relevance , 2013, SIGIR.

[5]  Yiqun Liu,et al.  From Skimming to Reading: A Two-stage Examination Model for Web Search , 2014, CIKM.

[6]  Nick Craswell,et al.  An experimental comparison of click position-bias models , 2008, WSDM '08.

[7]  Mark Levene,et al.  Presentation bias is significant in determining user preference for search results - A user study , 2009, J. Assoc. Inf. Sci. Technol..

[8]  Yiqun Liu,et al.  Incorporating vertical results into search click models , 2013, SIGIR.

[9]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[10]  Tamás D. Gedeon,et al.  Eye‐tracking analysis of user behavior and performance in web search on large and small screens , 2015, J. Assoc. Inf. Sci. Technol..

[11]  Alexander J. Smola,et al.  Measurement and modeling of eye-mouse behavior in the presence of nonlinear page layouts , 2013, WWW.

[12]  Yisong Yue,et al.  Beyond position bias: examining result attractiveness as a source of presentation bias in clickthrough data , 2010, WWW '10.

[13]  Thorsten Joachims,et al.  Accurately Interpreting Clickthrough Data as Implicit Feedback , 2017 .

[14]  Yiqun Liu,et al.  Understanding Reading Attention Distribution during Relevance Judgement , 2018, CIKM.

[15]  José Luis Vicedo González,et al.  TREC: Experiment and evaluation in information retrieval , 2007, J. Assoc. Inf. Sci. Technol..

[16]  M. de Rijke,et al.  Click Models for Web Search , 2015, Click Models for Web Search.

[17]  Chirag Shah,et al.  Discrimination between tasks with user activity patterns during information search , 2014, SIGIR.

[18]  Vidhya Navalpakkam,et al.  Understanding Mobile Searcher Attention with Rich Ad Formats , 2016, CIKM.

[19]  Shaoping Ma,et al.  Constructing Click Models for Mobile Search , 2018, SIGIR.

[20]  M. de Rijke,et al.  A Neural Click Model for Web Search , 2016, WWW.

[21]  Matthew Richardson,et al.  Predicting clicks: estimating the click-through rate for new ads , 2007, WWW '07.

[22]  Eugene Agichtein,et al.  Beyond dwell time: estimating document relevance from cursor movements and other post-click searcher behavior , 2012, WWW.

[23]  Susan T. Dumais,et al.  The good, the bad, and the random: an eye-tracking study of ad quality in web search , 2010, SIGIR.

[24]  Fan Zhang,et al.  Evaluating Mobile Search with Height-Biased Gain , 2017, SIGIR.

[25]  Nina Mishra,et al.  Domain bias in web search , 2012, WSDM '12.

[26]  Ellen M. Voorhees,et al.  TREC: Experiment and Evaluation in Information Retrieval (Digital Libraries and Electronic Publishing) , 2005 .

[27]  Fernando Diaz,et al.  Search Result Prefetching on Desktop and Mobile , 2017, TOIS.

[28]  KimJaewon,et al.  Eye-tracking analysis of user behavior and performance in web search on large and small screens , 2015 .

[29]  Yang Song,et al.  Exploring and exploiting user search behavior on mobile and tablet devices to improve search relevance , 2013, WWW '13.

[30]  Yiqun Liu,et al.  Influence of Vertical Result in Web Search Examination , 2015, SIGIR.

[31]  Milad Shokouhi,et al.  From Queries to Cards: Re-ranking Proactive Card Recommendations Based on Reactive Search History , 2015, SIGIR.

[32]  E. Gordon,et al.  Defining the temporal threshold for ocular fixation in free-viewing visuocognitive tasks , 2003, Journal of Neuroscience Methods.

[33]  Yiqun Liu,et al.  A Large-Scale Study of Mobile Search Examination Behavior , 2018, SIGIR.

[34]  Susan T. Dumais,et al.  Individual differences in gaze patterns for web search , 2010, IIiX.

[35]  Edward Cutrell,et al.  An eye tracking study of the effect of target rank on web search , 2007, CHI.

[36]  Chih-Hung Hsieh,et al.  Towards better measurement of attention and satisfaction in mobile search , 2014, SIGIR.

[37]  Yiqun Liu,et al.  Constructing Click Model for Mobile Search with Viewport Time , 2019, ACM Trans. Inf. Syst..

[38]  Thorsten Joachims,et al.  Eye-tracking analysis of user behavior in WWW search , 2004, SIGIR '04.

[39]  J. Fleiss.  Measuring nominal scale agreement among many raters. , 1971 .

[40]  Olivier Chapelle,et al.  A dynamic bayesian network click model for web search ranking , 2009, WWW '09.