Multi-Session Diversity to Improve User Satisfaction in Web Applications

In various Web applications, users consume content in a series of sessions. That is prevalent in online music listening, where a session is a channel and channels are listened to in sequence, or in crowdsourcing, where a session is a set of tasks and task sets are completed in sequence. Content diversity can be defined in more than one way, e.g., based on artists or genres for music, or on requesters or rewards in crowdsourcing. A user may prefer to experience diversity within or across sessions. Naturally, intra-session diversity is set-based, whereas, inter-session diversity is sequence-based. This novel multi-session diversity gives rise to four bi-objective problems with the goal of minimizing or maximizing inter and intra diversities. Given the hardness of those problems, we propose to formulate a constrained optimization problem that optimizes inter diversity, subject to the constraint of intra diversity. We develop an efficient algorithm to solve our problem. Our experiments with human subjects on two real datasets, music and crowdsourcing, show our diversity formulations do serve different user needs, and yield high user satisfaction. Our large data experiments on real and synthetic data empirically demonstrate that our solution satisfy the theoretical bounds and is highly scalable, compared to baselines.

[1]  Dana Chandler,et al.  Breaking Monotony with Meaning: Motivation in Crowdsourcing Markets , 2012, ArXiv.

[2]  Lei Han,et al.  All Those Wasted Hours: On Task Abandonment in Crowdsourcing , 2019, WSDM.

[3]  Reynold Cheng,et al.  QASCA: A Quality-Aware Task Assignment System for Crowdsourcing Applications , 2015, SIGMOD Conference.

[4]  M. R. Stoline The Status of Multiple Comparisons: Simultaneous Estimation of all Pairwise Comparisons in One-Way ANOVA Designs , 1981 .

[5]  Chien-Ju Ho,et al.  Adaptive Task Assignment for Crowdsourced Classification , 2013, ICML.

[6]  Beng Chin Ooi,et al.  iCrowd: An Adaptive Crowdsourcing Framework , 2015, SIGMOD Conference.

[7]  Michael S. Bernstein,et al.  A Glimpse Far into the Future: Understanding Long-term Crowd Worker Quality , 2016, CSCW.

[8]  Jade Goldstein-Stewart,et al.  The use of MMR, diversity-based reranking for reordering documents and producing summaries , 1998, SIGIR '98.

[9]  Panagiotis G. Ipeirotis,et al.  Demographics and Dynamics of Mechanical Turk Workers , 2018, WSDM.

[10]  Scott Sanner,et al.  Two-stage Model for Automatic Playlist Continuation at Scale , 2018, RecSys Challenge.

[11]  Saul Vargas,et al.  Coverage, redundancy and size-awareness in genre diversity for recommender systems , 2014, RecSys '14.

[12]  Xiaoyan Zhu,et al.  Promoting Diversity in Recommendation by Entropy Regularizer , 2013, IJCAI.

[13]  Jayant R. Haritsa,et al.  Providing Diversity in K-Nearest Neighbor Query Results , 2003, PAKDD.

[14]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[15]  Sihem Amer-Yahia,et al.  Personalized and Diverse Task Composition in Crowdsourcing , 2018, IEEE Transactions on Knowledge and Data Engineering.

[16]  Dafna Shahaf,et al.  Turning down the noise in the blogosphere , 2009, KDD.

[17]  Guandong Xu,et al.  Sequence-based context-aware music recommendation , 2018, Information Retrieval Journal.

[18]  Cong Yu,et al.  It takes variety to make a world: diversification in recommender systems , 2009, EDBT '09.

[19]  M. L. Fisher,et al.  An analysis of approximations for maximizing submodular set functions—I , 1978, Math. Program..

[20]  Sihem Amer-Yahia,et al.  Motivation-Aware Task Assignment in Crowdsourcing , 2017, EDBT.

[21]  Gianluca Demartini,et al.  Scaling-Up the Crowd: Micro-Task Pricing Schemes for Worker Retention and Latency Improvement , 2014, HCOMP.

[22]  Nick Koudas,et al.  Efficient diversity-aware search , 2011, SIGMOD '11.

[23]  Sihem Amer-Yahia,et al.  Real-time recommendation of diverse related articles , 2013, WWW.

[24]  Mi Zhang,et al.  Avoiding monotony: improving the diversity of recommendation lists , 2008, RecSys '08.

[25]  Beng Chin Ooi,et al.  A hybrid machine-crowdsourcing system for matching web tables , 2014, 2014 IEEE 30th International Conference on Data Engineering.

[26]  Abraham P. Punnen,et al.  TSP Heuristics: Domination Analysis and Complexity , 2003, Algorithmica.

[27]  Thierry Bertin-Mahieux,et al.  The Million Song Dataset , 2011, ISMIR.

[28]  Andrei Z. Broder,et al.  Sampling Search-Engine Results , 2005, WWW '05.

[29]  Chien-Ju Ho,et al.  Online Task Assignment in Crowdsourcing Markets , 2012, AAAI.

[30]  Sihem Amer-Yahia,et al.  Task Composition in Crowdsourcing , 2016, 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[31]  Yves Grandvalet,et al.  A Coverage-Based Approach to Recommendation Diversity On Similarity Graph , 2016, RecSys.

[32]  Sean M. McNee,et al.  Improving recommendation lists through topic diversification , 2005, WWW '05.

[33]  Tao Li,et al.  Addressing diverse user preferences in SQL-query-result navigation , 2007, SIGMOD '07.

[34]  Sihem Amer-Yahia,et al.  Optimized group formation for solving collaborative tasks , 2018, The VLDB Journal.

[35]  Ujwal Gadiraju,et al.  SimilarHITs: Revealing the Role of Task Similarity in Microtask Crowdsourcing , 2018, HT.

[36]  Peng Dai,et al.  And Now for Something Completely Different: Improving Crowdsourcing Workflows with Micro-Diversions , 2015, CSCW.

[37]  Robin Burke,et al.  Context-aware music recommendation based on latenttopic sequential patterns , 2012, RecSys.