Exploring author gender in book rating and recommendation

Collaborative filtering algorithms find useful patterns in rating and consumption data and exploit these patterns to guide users to good items. Many of the patterns in rating datasets reflect important real-world differences between the various users and items in the data; other patterns may be irrelevant or possibly undesirable for social or ethical reasons, particularly if they reflect undesired discrimination, such as gender or ethnic discrimination in publishing. In this work, we examine the response of collaborative filtering recommender algorithms to the distribution of their input data with respect to a dimension of social concern, namely content creator gender. Using publicly-available book ratings data, we measure the distribution of the genders of the authors of books in user rating profiles and recommendation lists produced from this data. We find that common collaborative filtering algorithms differ in the gender distribution of their recommendation lists, and in the relationship of that output distribution to user profile distribution.

[1]  Morgan Klaus Scheuerman,et al.  Gender Recognition or Gender Reductionism?: The Social Implications of Embedded Gender Recognition Systems , 2018, CHI.

[2]  Carlos Eduardo Scheidegger,et al.  Certifying and Removing Disparate Impact , 2014, KDD.

[3]  John Riedl,et al.  An algorithmic framework for performing collaborative filtering , 1999, SIGIR '99.

[4]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[5]  Kelly A. Byrne,et al.  R2: Dell HPC Intel E5v4 (High Performance Computing Cluster) , 2017 .

[6]  Alejandro Bellogín,et al.  Precision-oriented evaluation of recommender systems: an algorithmic comparison , 2011, RecSys '11.

[7]  Sean M. McNee,et al.  Improving recommendation lists through topic diversification , 2005, WWW '05.

[8]  Toniann Pitassi,et al.  Fairness through awareness , 2011, ITCS '12.

[9]  Alex Rosenblat,et al.  Algorithmic Labor and Information Asymmetries: A Case Study of Uber’s Drivers , 2016 .

[10]  P KnijnenburgBart,et al.  Explaining the user experience of recommender systems , 2012 .

[11]  Loren G. Terveen,et al.  Exploring the filter bubble: the effect of using recommender systems on content diversity , 2014, WWW.

[12]  Helen Nissenbaum,et al.  Bias in computer systems , 1996, TOIS.

[13]  Bart P. Knijnenburg,et al.  Explaining the user experience of recommender systems , 2012, User Modeling and User-Adapted Interaction.

[14]  David M. Blei,et al.  Scalable Recommendation with Poisson Factorization , 2013, ArXiv.

[15]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[16]  Francis Tuerlinckx,et al.  Type S error rates for classical and Bayesian single and multiple comparison procedures , 2000, Comput. Stat..

[17]  David García,et al.  Bias in Online Freelance Marketplaces: Evidence from TaskRabbit and Fiverr , 2017, CSCW.

[18]  Saul Vargas,et al.  Rank and relevance in novelty and diversity metrics for recommender systems , 2011, RecSys '11.

[19]  Neil J. Hurley,et al.  Novelty and Diversity in Top-N Recommendation -- Analysis and Evaluation , 2011, TOIT.

[20]  Virgílio A. F. Almeida,et al.  Stereotypes in Search Engine Results: Understanding The Role of Local and Global Factors , 2016, 1609.05413.

[21]  Jun Sakuma,et al.  Recommendation Independence , 2018, FAT.

[22]  K. Lum,et al.  To predict and serve? , 2016 .

[23]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[24]  John Riedl,et al.  Rethinking the recommender research ecosystem: reproducibility, openness, and LensKit , 2011, RecSys '11.

[25]  F. Maxwell Harper,et al.  The MovieLens Datasets: History and Context , 2016, TIIS.

[26]  Bert Huang,et al.  Beyond Parity: Fairness Objectives for Collaborative Filtering , 2017, NIPS.

[27]  Sune Lehmann,et al.  Understanding the Demographics of Twitter Users , 2011, ICWSM.

[28]  Maria Soledad Pera,et al.  All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Recommender Evaluation and Effectiveness , 2018, FAT.

[29]  Harald Steck,et al.  Calibrated recommendations , 2018, RecSys.

[30]  Mark P. Graus,et al.  Understanding the role of latent feature diversification on choice difficulty and satisfaction , 2016, User Modeling and User-Adapted Interaction.

[31]  Erik Brynjolfsson,et al.  Global Village or Cyberbalkans: Modeling and Measuring the Integration of Electronic Communities , 2005, Manag. Sci..

[32]  Martijn C. Willemsen,et al.  Behaviorism is Not Enough: Better Recommendations through Listening to Users , 2016, RecSys.

[33]  Nasim Sonboli,et al.  Balanced Neighborhoods for Multi-sided Fairness in Recommendation , 2018, FAT.

[34]  Robin D. Burke,et al.  Multisided Fairness for Recommendation , 2017, ArXiv.

[35]  Timnit Gebru,et al.  Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification , 2018, FAT.

[36]  Karl J. Friston,et al.  Hierarchical Models , 2003 .

[37]  George Karypis,et al.  Item-based top-N recommendation algorithms , 2004, TOIS.

[38]  P. Resnick Beyond Bowling Together: SocioTechnical Capital , 2001 .

[39]  Dietmar Jannach,et al.  What recommenders recommend: an analysis of recommendation biases and possible countermeasures , 2015, User Modeling and User-Adapted Interaction.

[40]  Mylène Bédard,et al.  Hierarchical models: Local proposal variances for RWM-within-Gibbs and MALA-within-Gibbs , 2017, Comput. Stat. Data Anal..

[41]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[42]  Suresh Venkatasubramanian,et al.  On the (im)possibility of fairness , 2016, ArXiv.

[43]  Guy Shani,et al.  Evaluating Recommendation Systems , 2011, Recommender Systems Handbook.

[44]  Ed H. Chi,et al.  Fairness in Recommendation Ranking through Pairwise Comparisons , 2019, KDD.

[45]  Martijn C. Willemsen,et al.  Effective User Interface Designs to Increase Energy-efficient Behavior in a Rasch-based Energy Recommender System , 2017, RecSys.

[46]  Òscar Celma,et al.  Music recommendation and discovery in the long tail , 2008 .

[47]  John Riedl,et al.  SuggestBot: using intelligent task routing to help people find work in wikipedia , 2007, IUI '07.

[48]  Piotr Sapiezynski,et al.  Quantifying the Impact of User Attentionon Fair Group Representation in Ranked Lists , 2019, WWW.

[49]  Mengting Wan,et al.  Item recommendation on monotonic behavior chains , 2018, RecSys.

[50]  Michael D. Ekstrand The LKPY Package for Recommender Systems Experiments: Next-Generation Tools and Lessons Learned from the LensKit Project , 2018, ArXiv.

[51]  Licia Capra,et al.  Temporal diversity in recommender systems , 2010, SIGIR.

[52]  Domonkos Tikk,et al.  Fast als-based matrix factorization for explicit and implicit feedback datasets , 2010, RecSys '10.

[53]  Adam Tauman Kalai,et al.  Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings , 2016, NIPS.

[54]  Julia Stoyanovich,et al.  Measuring Fairness in Ranked Outputs , 2016, SSDBM.

[55]  Christopher J. Riederer,et al.  The Price of Fairness in Location Based Advertising , 2017 .

[56]  Michael D. Ekstrand,et al.  Recommender Response to Diversity and Popularity Bias in User Profiles , 2017, FLAIRS.

[57]  Krishna P. Gummadi,et al.  Equity of Attention: Amortizing Individual Fairness in Rankings , 2018, SIGIR.

[58]  John Riedl,et al.  Collaborative Filtering Recommender Systems , 2011, Found. Trends Hum. Comput. Interact..

[59]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[60]  Eli Pariser,et al.  The Filter Bubble: How the New Personalized Web Is Changing What We Read and How We Think , 2012 .

[61]  Suresh Venkatasubramanian,et al.  Runaway Feedback Loops in Predictive Policing , 2017, FAT.