Understanding User Attention and Engagement in Online News Reading

Prior work on user engagement with online media identified web page dwell time as a key metric reflecting level of user engagement with online news articles. While on average, dwell time gives a reasonable estimate of user experience with a news article, it is not able to capture important aspects of user interaction with the page, such as how much time a user spends reading the article vs. viewing the comment posted by other users, or the actual proportion of article read by the user. In this paper, we propose a set of user engagement classes along with new user engagement metrics that, unlike dwell time, more accurately reflect user experience with the content. Our user engagement classes provide clear and interpretable taxonomy of user engagement with online news, and are defined based on amount of time user spends on the page, proportion of the article user actually reads and the amount of interaction users performs with the comments. Moreover, we demonstrate that our metrics are relatively easier to predict from the news article content, compared to the dwell time, making optimization of user engagement more attainable goal.

[1]  J. Lafferty,et al.  Mixed-membership models of scientific publications , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[2]  Eugene Agichtein,et al.  Improving search result summaries by using searcher behavior data , 2013, SIGIR.

[3]  Ryen W. White,et al.  Mining the search trails of surfing crowds: identifying relevant websites from user activity , 2008, WWW.

[4]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[5]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[6]  John R. Anderson,et al.  What can a mouse cursor tell us more?: correlation of eye/mouse movements on web browsing , 2001, CHI Extended Abstracts.

[7]  Bracha Shapira,et al.  Study of the usefulness of known and new implicit indicators and their optimal combination for accurate inference of users interests , 2006, SAC.

[8]  Yoichi Shinoda,et al.  Information filtering based on user behavior analysis and best match text retrieval , 1994, SIGIR '94.

[9]  Mark Claypool,et al.  Implicit interest indicators , 2001, IUI '01.

[10]  Joemon M. Jose,et al.  User engagement in online News: Under the scope of sentiment, interest, affect, and gaze , 2014, J. Assoc. Inf. Sci. Technol..

[11]  Gabriella Kazai,et al.  Towards a science of user engagement. , 2011 .

[12]  Thorsten Joachims,et al.  Eye tracking and online search: Lessons learned and challenges ahead , 2008 .

[13]  Eric Brill,et al.  Improving web search ranking by incorporating user behavior information , 2006, SIGIR.

[14]  Andrew McCallum,et al.  Topic Models Conditioned on Arbitrary Features with Dirichlet-multinomial Regression , 2008, UAI.

[15]  Steve Fox,et al.  Evaluating implicit measures to improve web search , 2005, TOIS.

[16]  Ryen W. White,et al.  Understanding web browsing behaviors through Weibull analysis of dwell time , 2010, SIGIR.

[17]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[18]  Ryen W. White,et al.  Modeling dwell time to predict click-level satisfaction , 2014, WSDM.

[19]  Eugene Agichtein,et al.  ViewSer: enabling large-scale remote user studies of web search examination and interaction , 2011, SIGIR.

[20]  Nick Craswell,et al.  An experimental comparison of click position-bias models , 2008, WSDM '08.

[21]  Chih-Hung Hsieh,et al.  Towards better measurement of attention and satisfaction in mobile search , 2014, SIGIR.

[22]  Mounia Lalmas,et al.  On saliency, affect and focused attention , 2012, CHI.

[23]  Alexander J. Smola,et al.  Measurement and modeling of eye-mouse behavior in the presence of nonlinear page layouts , 2013, WWW.

[24]  Suju Rajan,et al.  Beyond clicks: dwell time for personalization , 2014, RecSys '14.

[25]  Jude W. Shavlik,et al.  Learning users' interests by unobtrusively observing their normal behavior , 2000, IUI '00.

[26]  E. Toms,et al.  The development and evaluation of a survey to measure user engagement , 2010, J. Assoc. Inf. Sci. Technol..

[27]  George Valkanas,et al.  Understanding Within-Content Engagement through Pattern Analysis of Mouse Gestures , 2014, CIKM.

[28]  Eugene Agichtein,et al.  Discovering common motifs in cursor movement data for improving web search , 2014, WSDM.

[29]  Andrew McCallum,et al.  Topics over time: a non-Markov continuous-time model of topical trends , 2006, KDD '06.

[30]  Yunfei Chen,et al.  Evaluating the visual quality of web pages using a computational aesthetic approach , 2011, WSDM '11.

[31]  T. Minka.  Estimating a Dirichlet distribution , 2012 .

[32]  Andreas Dengel,et al.  Segment-level display time as implicit feedback: a comparison to eye tracking , 2009, SIGIR.

[33]  Eugene Agichtein,et al.  Beyond dwell time: estimating document relevance from cursor movements and other post-click searcher behavior , 2012, WWW.