Do not blame it on the algorithm: an empirical assessment of multiple recommender systems and their impact on content diversity

ABSTRACT In the debate about filter bubbles caused by algorithmic news recommendation, the conceptualization of the two core concepts in this debate, diversity and algorithms, has received little attention in social scientific research. This paper examines the effect of multiple recommender systems on different diversity dimensions. To this end, it maps different values that diversity can serve, and a respective set of criteria that characterizes a diverse information offer in this particular conception of diversity. We make use of a data set of simulated article recommendations based on actual content of one of the major Dutch broadsheet newspapers and its users (N=21,973 articles, N=500 users). We find that all of the recommendation logics under study proved to lead to a rather diverse set of recommendations that are on par with human editors and that basing recommendations on user histories can substantially increase topic diversity within a recommendation set.

[1]  Sean M. McNee,et al.  Improving recommendation lists through topic diversification , 2005, WWW '05.

[2]  Philip M. Napoli.  Assessing Media Diversity in the U.S.: A Comparative Analysis of the FCC’s Diversity Index and the EU’s Media Pluralism Monitor , 2015 .

[3]  Jesper Strömbäck,et al.  In Search of a Standard: four models of democracy and their normative implications for journalism , 2005 .

[4]  Lior Rokach,et al.  Introduction to Recommender Systems Handbook , 2011, Recommender Systems Handbook.

[5]  R. Kitchin,et al.  Thinking critically about and researching algorithms , 2014, The Social Power of Algorithms.

[6]  Justin M. Rao,et al.  Filter Bubbles, Echo Chambers, and Online News Consumption , 2016 .

[7]  Walter Daelemans,et al.  Pattern for Python , 2012, J. Mach. Learn. Res..

[8]  Derek Greene,et al.  Down the (White) Rabbit Hole: The Extreme Right and Online Recommender Systems , 2015 .

[9]  Ivan B. Dylko.  How Technology Encourages Political Selective Exposure , 2016 .

[10]  N. Stroud.  Niche News: The Politics of News Choice , 2011 .

[11]  Jannick Kirk Sørensen,et al.  An Algorithmic Diversity Diet?: Questioning Assumptions behind a Diversity Recommendation System for PSM , 2016 .

[12]  Engin Bozdag,et al.  Bias in algorithmic filtering and personalization , 2013, Ethics and Information Technology.

[13]  K. Karppinen.  Rethinking Media Pluralism , 2013 .

[14]  Barry Smyth,et al.  Similarity vs. Diversity , 2001, ICCBR.

[15]  Mark P. Graus,et al.  Understanding the role of latent feature diversification on choice difficulty and satisfaction , 2016, User Modeling and User-Adapted Interaction.

[16]  Matevz Kunaver,et al.  Diversity in recommender systems - A survey , 2017, Knowl. Based Syst..

[17]  Lucia D'Acunto,et al.  Exposure diversity as a design principle for recommender systems , 2018 .

[18]  C. Ricotta,et al.  Towards a unifying approach to diversity measures: bridging the gap between the Shannon entropy and Rao's quadratic index. , 2006, Theoretical population biology.

[19]  M. de Rijke,et al.  Hierarchical Re-estimation of Topic Models for Measuring Topical Diversity , 2017, ECIR.

[20]  Shuaiqiang Wang,et al.  A survey of serendipity in recommender systems , 2016, Knowl. Based Syst..

[21]  David Owen,et al.  Survey Article: Deliberation, Democracy, and the Systemic Turn , 2015 .

[22]  Roland Schäfer,et al.  Building Large Corpora from the Web Using a New Efficient Tool Chain , 2012, LREC.

[23]  Anísio Lacerda,et al.  Multiobjective Pareto-Efficient Approaches for Recommender Systems , 2014, ACM Trans. Intell. Syst. Technol..

[24]  Jürgen Gerhards,et al.  Four models of the public sphere in modern democracies , 2002 .

[25]  Walter Daelemans,et al.  Media coverage in times of political crisis: A text mining approach , 2012, Expert Syst. Appl..

[26]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[27]  Bart P. Knijnenburg,et al.  Explaining the user experience of recommender systems , 2012, User Modeling and User-Adapted Interaction.

[28]  Derek G. Bridge,et al.  Ways of Computing Diverse Collaborative Recommendations , 2006, AH.

[29]  Ivan B. Dylko,et al.  The dark side of technology: An experimental investigation of the influence of customizability technology on online political selective exposure , 2017, Comput. Hum. Behav..

[30]  R. Nielsen,et al.  Are News Audiences Increasingly Fragmented? A Cross-National Comparative Analysis of Cross-Platform News Audience Fragmentation and Duplication , 2017 .

[31]  Kenneth E. Shirley,et al.  LDAvis: A method for visualizing and interpreting topics , 2014 .

[32]  Eli Pariser,et al.  The Filter Bubble: What the Internet Is Hiding from You , 2011 .

[33]  Pamela J. Shoemaker,et al.  Individual and Routine Forces in Gatekeeping , 2001 .

[34]  Rodney Benson,et al.  What makes news more multiperspectival? A field analysis , 2009 .

[35]  Lada A. Adamic,et al.  Exposure to ideologically diverse news and opinion on Facebook , 2015, Science.

[36]  Jane Duncan,et al.  Toward a measurement tool for the monitoring of media diversity and pluralism in South Africa: A public-centred approach , 2013 .

[37]  Petr Sojka,et al.  Software Framework for Topic Modelling with Large Corpora , 2010 .

[38]  S. Adee.  Burst the filter bubble , 2016 .

[39]  Kari Karppinen,et al.  The Limits of Empirical Indicators: Media Pluralism as an Essentially Contested Concept , 2015 .

[40]  Damian Trilling,et al.  Should We Worry About Filter Bubbles? , 2016 .

[41]  Walter Daelemans,et al.  Evaluating Unsupervised Dutch Word Embeddings as a Linguistic Resource , 2016, LREC.

[42]  Andrea Masini,et al.  Measuring and Explaining the Diversity of Voices and Viewpoints in the News , 2017 .

[43]  Jan van Cuilenburg.  On Competition, Access and Diversity in Media, Old and New: Some Remarks for Communications Policy in the Information Age , 1999, New Media Soc..