Reconciling the Quality vs Popularity Dichotomy in Online Cultural Markets

We propose a simple model of an idealized online cultural market in which N items, endowed with a hidden quality metric, are recommended to users by a ranking algorithm possibly biased by the current items’ popularity. Our goal is to better understand the underlying mechanisms of the well-known fact that popularity bias can prevent higher-quality items from becoming more popular than lower-quality items, producing an undesirable misalignment between quality and popularity rankings. We do so under the assumption that users, having limited time/attention, are able to discriminate the best-quality only within a random subset of the items. We discover the existence of a harmful regime in which improper use of popularity can seriously compromise the emergence of quality, and a benign regime in which wise use of popularity, coupled with a small discrimination effort on behalf of users, guarantees the perfect alignment of quality and popularity ranking. Our findings clarify the effects of algorithmic popularity bias on quality outcomes, and may inform the design of more principled mechanisms for techno-social cultural markets.

[1]  Pascal Van Hentenryck,et al.  Aligning Popularity and Quality in Online Cultural Markets , 2021, ICWSM.

[2]  Robin Burke,et al.  The Connection Between Popularity Bias, Calibration, and Fairness in Recommendation , 2020, RecSys.

[3]  Mark Sanderson,et al.  Agreement and Disagreement between True and False-Positive Metrics in Recommender Systems Evaluation , 2020, SIGIR.

[4]  Bamshad Mobasher,et al.  Feedback Loop and Bias Amplification in Recommender Systems , 2020, CIKM.

[5]  Alexander D'Amour,et al.  Fairness is not static: deeper understanding of long term fairness via simulation studies , 2020, FAT*.

[6]  Robin Burke,et al.  The Unfairness of Popularity Bias in Recommendation , 2019, RMSE@RecSys.

[7]  Himan Abdollahpouri,et al.  Popularity Bias in Ranking and Recommendation , 2019, AIES.

[8]  Tor Lattimore,et al.  Degenerate Feedback Loops in Recommender Systems , 2019, AIES.

[9]  Pablo Castells,et al.  Should I Follow the Crowd?: A Probabilistic Analysis of the Effectiveness of Popularity in Recommender Systems , 2018, SIGIR.

[10]  Barbara E. Engelhardt,et al.  How algorithmic confounding in recommendation systems increases homogeneity and decreases utility , 2017, RecSys.

[11]  Bamshad Mobasher,et al.  Controlling Popularity Bias in Learning-to-Rank Recommendation , 2017, RecSys.

[12]  Filippo Menczer,et al.  How algorithmic popularity bias hinders or promotes quality , 2017, Scientific Reports.

[13]  Guandong Xu,et al.  Improving the Quality of Recommendations for Users and Items in the Tail of Distribution , 2017, ACM Trans. Inf. Syst..

[14]  Alireza Sahami Shirazi,et al.  Limited individual attention and online virality of low-quality information , 2017, Nature Human Behaviour.

[15]  Dietmar Jannach,et al.  What recommenders recommend: an analysis of recommendation biases and possible countermeasures , 2015, User Modeling and User-Adapted Interaction.

[16]  Greg Stoddard,et al.  Popularity Dynamics and Intrinsic Quality in Reddit and Hacker News , 2015, ICWSM.

[17]  G. Viglia,et al.  Please, talk about it! When hotel popularity boosts preferences☆ , 2014 .

[18]  Soong Moon Kang,et al.  Field experiments of success-breeds-success dynamics , 2014, Proceedings of the National Academy of Sciences.

[19]  Sean J. Taylor,et al.  Social Influence Bias: A Randomized Experiment , 2013, Science.

[20]  Eskil Forsell,et al.  Demand Effects of Consumers' Stated and Revealed Preferences , 2013 .

[21]  Jeff Manza.  Everything is Obvious Once You Know the Answer , 2012, European Journal of Sociology.

[22]  Galen Pickard,et al.  Quantifying Social Influence in an Online Cultural Market , 2012, PloS one.

[23]  A. Vespignani,et al.  Competition among memes in a world with limited attention , 2012, Scientific Reports.

[24]  Susan K. Walker.  Connected: The Surprising Power of Our Social Networks and How They Shape Our Lives , 2011 .

[25]  D. Helbing,et al.  How social influence can undermine the wisdom of crowd effect , 2011, Proceedings of the National Academy of Sciences.

[26]  D. Watts.  Everything Is Obvious: *Once You Know the Answer , 2011 .

[27]  Duncan J. Watts,et al.  Everyone's an influencer: quantifying influence on twitter , 2011, WSDM '11.

[28]  Bracha Shapira,et al.  Recommender Systems Handbook , 2015, Springer US.

[29]  D. Rigney,et al.  The Matthew Effect: How Advantage Begets Further Advantage , 2011 .

[30]  Richard S. Zemel,et al.  Collaborative prediction and ranking with non-random missing data , 2009, RecSys '09.

[31]  Nick Chater,et al.  Herding in humans , 2009, Trends in Cognitive Sciences.

[32]  Matthew J. Salganik,et al.  Web-Based Experiments for the Study of Collective Social Dynamics in Cultural Markets , 2009, Top. Cogn. Sci..

[33]  Matthew J. Salganik,et al.  Leading the Herd Astray: An Experimental Study of Self-fulfilling Prophecies in an Artificial Cultural Market , 2008, Social psychology quarterly.

[34]  Hosam M. Mahmoud,et al.  Polya Urn Models , 2008 .

[35]  Matthew J. Salganik,et al.  Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market , 2006, Science.

[36]  Santo Fortunato,et al.  Scale-free network growth by ranking. , 2006, Physical review letters.

[37]  Santo Fortunato,et al.  Topical interests and the mitigation of search engine bias , 2005, Proceedings of the National Academy of Sciences.

[38]  Junghoo Cho,et al.  Impact of search engines on page popularity , 2004, WWW '04.

[39]  Mathew D. Penrose,et al.  Random Geometric Graphs , 2003 .

[40]  Michael Mitzenmacher,et al.  The Power of Two Choices in Randomized Load Balancing , 2001, IEEE Trans. Parallel Distributed Syst..

[41]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[42]  Eli Upfal,et al.  Balanced Allocations , 1999, SIAM J. Comput..

[43]  A. Banerjee,et al.  A Simple Model of Herd Behavior , 1992 .

[44]  I. Welch.  Sequential Sales, Learning, and Cascades , 1992 .

[45]  S. Bikhchandani,et al.  You have printed the following article : A Theory of Fads , Fashion , Custom , and Cultural Change as Informational Cascades , 2007 .

[46]  H. Simon,et al.  ON A CLASS OF SKEW DISTRIBUTION FUNCTIONS , 1955 .

[47]  C. Gershenson,et al.  Universal dynamics of ranking , 2021 .

[48]  Michael Vitale,et al.  The wisdom of crowds , 2016, The Lancet.

[49]  Please, talk about it , 2015 .

[50]  Yehuda Koren,et al.  Recommender Systems Handbook , 2011, Recommender Systems Handbook.

[51]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[52]  George Karypis,et al.  Item-based top-N recommendation algorithms , 2004, TOIS.

[53]  Ramesh K. Sitaraman,et al.  The power of two random choices: a survey of tech-niques and results , 2001 .

[54]  P. Todd,et al.  Simple Heuristics That Make Us Smart , 1999 .

[55]  R. Merton.  The Matthew effect in science. The reward and communication systems of science are considered. , 1968, Science.