Algorithms are not neutral

Discussions of algorithmic bias tend to focus on examples where either the data or the people building the algorithms are biased. This gives the impression that clean data and good intentions could eliminate bias. The neutrality of the algorithms themselves is defended by prominent Artificial Intelligence researchers. However, algorithms are not neutral. In addition to biased data and biased algorithm makers, AI algorithms themselves can be biased. This is illustrated with the example of collaborative filtering, which is known to suffer from popularity, and homogenizing biases. Iterative information filtering algorithms in general create a selection bias in the course of learning from user responses to documents that the algorithm recommended. These are not merely biases in the statistical sense; these statistical biases can cause discriminatory outcomes. Data points on the margins of distributions of human data tend to correspond to marginalized people. Popularity and homogenizing biases have the effect of further marginalizing the already marginal. This source of bias warrants serious attention given the ubiquity of algorithmic decision-making.

[1]  Maria Soledad Pera,et al.  All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Recommender Evaluation and Effectiveness , 2018, FAT.

[2]  Engin Bozdag,et al.  Bias in algorithmic filtering and personalization , 2013, Ethics and Information Technology.

[3]  Alexander Tuzhilin,et al.  On over-specialization and concentration bias of recommendations: probabilistic neighborhood selection in collaborative filtering systems , 2014, RecSys '14.

[4]  Helen Nissenbaum,et al.  Bias in computer systems , 1996, TOIS.

[5]  Nitesh V. Chawla,et al.  Learning From Labeled And Unlabeled Data: An Empirical Study Across Techniques And Domains , 2011, J. Artif. Intell. Res..

[6]  Timnit Gebru,et al.  Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification , 2018, FAT.

[7]  Michael Runcieman.  YouTube, the Great Radicalizer - The New York Times , 2018 .

[8]  D. Fitch,et al.  Review of "Algorithms of oppression: how search engines reinforce racism," by Noble, S. U. (2018). New York, New York: NYU Press. , 2018, CDQR.

[9]  Emre Kıcıman,et al.  Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries , 2018, Front. Big Data.

[10]  Krishna P. Gummadi,et al.  Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment , 2016, WWW.

[11]  Fernando Diaz,et al.  Auditing Search Engines for Differential Satisfaction Across Demographics , 2017, WWW.

[12]  Jevin D. West,et al.  Echo Chambers in Science? , 2017 .

[13]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[14]  Andrew D. Selbst,et al.  Big Data's Disparate Impact , 2016 .

[15]  Kathy L. Powers,et al.  Regarding Docket No. FR-6111-P-02, HUD’s Implementation of the Fair Housing Act’s Disparate Impact Standard , 2019 .

[16]  Harald Steck,et al.  Item popularity and recommendation accuracy , 2011, RecSys '11.

[17]  Wei Chen,et al.  Opinion-Based Collaborative Filtering to Solve Popularity Bias in Recommender Systems , 2013, DEXA.

[18]  Olfa Nasraoui,et al.  Iterated Algorithmic Bias in the Interactive Machine Learning Process of Information Filtering , 2018, KDIR.