Tutorial on Online User Engagement: Metrics and Optimization

User engagement plays a central role in companies operating online services, such as search engines, news portals, e-commerce sites, entertainment services, and social networks. A main challenge is to leverage collected knowledge about the daily online behavior of millions of users to understand what engages them short-term and more importantly long-term. Two critical steps of improving user engagement are metrics and their optimization. The most common way that engagement is measured is through various online metrics, acting as proxy measures of user engagement. This tutorial will review these metrics, their advantages and drawbacks, and their appropriateness to various types of online services. Once metrics are defined, how to optimize them will become the key issue. We will survey methodologies including machine learning models and experimental designs that are utilized to optimize these metrics via direct or indirect ways. As case studies, we will focus on four types of services, news, search, entertainment, and e-commerce.

[1]  Suju Rajan,et al.  Beyond clicks: dwell time for personalization , 2014, RecSys '14.

[2]  Ke Zhou,et al.  Predicting Pre-click Quality for Native Advertisements , 2016, WWW.

[3]  Mounia Lalmas,et al.  You must have clicked on this ad by mistake! Data-driven identification of accidental clicks on mobile ads with applications to advertiser cost discounting and click-through rate prediction , 2018, International Journal of Data Science and Analytics.

[4]  Mounia Lalmas,et al.  Models of user engagement , 2012, UMAP.

[5]  Mounia Lalmas,et al.  Understanding User Attention and Engagement in Online News Reading , 2016, WSDM.

[6]  Gary Marchionini,et al.  Synthesis Lectures on Information Concepts, Retrieval, and Services , 2009 .

[7]  Fabrizio Silvestri,et al.  Improving Post-Click User Engagement on Native Ads via Survival Analysis , 2016, WWW.

[8]  Fernando Diaz,et al.  Towards a Fair Marketplace: Counterfactual Evaluation of the trade-off between Relevance, Fairness & Satisfaction in Recommendation Systems , 2018, CIKM.

[9]  Mounia Lalmas,et al.  Measuring User Engagement , 2014, Measuring User Engagement.

[10]  Ricardo Baeza-Yates,et al.  Story‐focused reading in online news and its potential for user engagement , 2017, J. Assoc. Inf. Sci. Technol..

[11]  Mounia Lalmas,et al.  Absence time and user engagement: evaluating ranking functions , 2013, WSDM '13.

[12]  Mounia Lalmas,et al.  Deriving User- and Content-specific Rewards for Contextual Bandits , 2019, WWW.

[13]  Yizhou Sun,et al.  On Sampling Strategies for Neural Network-based Collaborative Filtering , 2017, KDD.

[14]  Qian Zhao,et al.  GB-CENT: Gradient Boosted Categorical Embedding and Numerical Trees , 2017, WWW.

[15]  Naren Ramakrishnan,et al.  A Gradient-based Adaptive Learning Framework for Efficient Personal Recommendation , 2017, RecSys.

[16]  Mounia Lalmas,et al.  Reader preferences and behavior on Wikipedia , 2014, HT.