Algorithmic injustice: a relational ethics approach

Summary It has become trivial to point out that algorithmic systems increasingly pervade the social sphere. Improved efficiency—the hallmark of these systems—drives their mass integration into day-to-day life. However, as a robust body of research in the area of algorithmic injustice shows, algorithmic systems, especially when used to sort and predict social outcomes, are not only inadequate but also perpetuate harm. In particular, a persistent and recurrent trend within the literature indicates that society's most vulnerable are disproportionally impacted. When algorithmic injustice and harm are brought to the fore, most of the solutions on offer (1) revolve around technical solutions and (2) do not center disproportionally impacted communities. This paper proposes a fundamental shift—from rational to relational—in thinking about personhood, data, justice, and everything in between, and places ethics as something that goes above and beyond technical solutions. Outlining the idea of ethics built on the foundations of relationality, this paper calls for a rethinking of justice and ethics as a set of broad, contingent, and fluid concepts and down-to-earth practices that are best viewed as a habit and not a mere methodology for data science. As such, this paper mainly offers critical examinations and reflection and not “solutions.”

[1]  Yoav Goldberg,et al.  Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them , 2019, NAACL-HLT.

[2]  Sara Ahmed,et al.  A phenomenology of whiteness , 2007 .

[3]  Lorraine Daston,et al.  Calculation and the Division of Labor, 1750-1950 , 2018 .

[4]  Paul Dourish,et al.  Postcolonial computing: a lens on design and development , 2010, CHI.

[5]  Thomas A. Klein,et al.  DYNAMICS IN ACTION: INTENTIONAL BEHAVIOR AS A COMPLEX SYSTEM , 2003 .

[6]  Hanne De Jaegher Loving and knowing: reflections for an engaged epistemology , 2019, Phenomenology and the Cognitive Sciences.

[7]  E. D. Paolo,et al.  Participatory sense-making , 2007 .

[8]  Obioma Nnaemeka Nego‐Feminism: Theorizing, Practicing, and Pruning Africa’s Way , 2004, Signs: Journal of Women in Culture and Society.

[9]  Meredith Broussard,et al.  Artificial Unintelligence: How Computers Misunderstand the World , 2018 .

[10]  Kevin Slavin Design as Participation , 2016 .

[11]  Michael E. Gardiner ‘The Incomparable Monster of Solipsism’: Bakhtin and Merleau-Ponty , 1998 .

[12]  Brian W. Powers,et al.  Dissecting racial bias in an algorithm used to manage the health of populations , 2019, Science.

[13]  Sorelle A. Friedler,et al.  Hiring by Algorithm: Predicting and Preventing Disparate Impact , 2016 .

[14]  Yanni Alexander Loukissas,et al.  All Data Are Local , 2019 .

[15]  Virginia E. Eubanks Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor , 2018 .

[16]  E. D. Paolo,et al.  Linguistic Bodies: The Continuity between Life and Language , 2018 .

[17]  Geoffrey C. Bowker,et al.  Enacting silence: Residual categories as a challenge for ethics, information systems, and communication , 2007, Ethics and Information Technology.

[18]  S. Harding Rethinking Standpoint Epistemology : What is « Strong Objectivity? » , 1992 .

[19]  J. Overhage,et al.  Sorting Things Out: Classification and Its Consequences , 2001, Annals of Internal Medicine.

[20]  Hannah Lebovits Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor , 2018, Public Integrity.

[21]  Solon Barocas,et al.  Mitigating Bias in Algorithmic Employment Screening: Evaluating Claims and Practices , 2019, SSRN Electronic Journal.

[22]  Sasha Costanza-Chock Design Justice: Towards an Intersectional Feminist Framework for Design Theory and Practice , 2018, DRS2018: Catalyst.

[23]  Phillipp Kaestner,et al.  Understanding Computers And Cognition A New Foundation For Design , 2016 .

[24]  D. Pager,et al.  Bayesian Bigot? Statistical Discrimination, Stereotypes, and Employer Decision Making , 2009, The Annals of the American Academy of Political and Social Science.

[25]  Daniel McQuillan,et al.  Data Science as Machinic Neoplatonism , 2017, Philosophy & Technology.

[26]  B. Love,et al.  The myth of computational level theory and the vacuity of rational analysis , 2011, Behavioral and Brain Sciences.

[27]  U. Hahn The Bayesian boom: good thing or bad? , 2014, Front. Psychol..

[28]  Morgan Klaus Scheuerman,et al.  Gender Recognition or Gender Reductionism?: The Social Implications of Embedded Gender Recognition Systems , 2018, CHI.

[29]  T. Bayes LII. An essay towards solving a problem in the doctrine of chances. By the late Rev. Mr. Bayes, F. R. S. communicated by Mr. Price, in a letter to John Canton, A. M. F. R. S , 1763, Philosophical Transactions of the Royal Society of London.

[30]  K. Lum,et al.  To predict and serve? , 2016 .

[31]  Judy Hoffman,et al.  Predictive Inequity in Object Detection , 2019, ArXiv.

[32]  Pratyusha Kalluri Don’t ask if artificial intelligence is good or fair, ask how it shifts power , 2020, Nature.

[33]  Timnit Gebru,et al.  Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification , 2018, FAT.

[34]  Terry Winograd,et al.  Understanding computers and cognition - a new foundation for design , 1987 .

[35]  D. Fitch,et al.  Review of "Algorithms of oppression: how search engines reinforce racism," by Noble, S. U. (2018). New York, New York: NYU Press. , 2018, CDQR.

[36]  Neide Mayumi Osada,et al.  Black Feminist Thought: Knowledge, Consciousness, and the Politics of Empowerment , 2008 .

[37]  Miriam Kyselo The body social: an enactive approach to the self , 2014, Front. Psychol..

[38]  Abeba Birhane,et al.  Towards decolonising computational sciences , 2020, Kvinder, Køn & Forskning.

[39]  James Mussell Raw Data is an Oxymoron , 2014 .

[40]  Antje Kirchner,et al.  Measuring the predictability of life outcomes with a scientific mass collaboration , 2020, Proceedings of the National Academy of Sciences.

[41]  Rachel Schutt,et al.  Doing Data Science , 2013 .

[42]  Sarojini Nadar,et al.  “Stories are data with Soul” – lessons from black feminist epistemology , 2014 .

[43]  Christina N. Harrington The forgotten margins , 2020, Interactions.

[44]  Heinz von Foerster,et al.  The Metaphysics of Ethics: A Conversation , 2002, Cybern. Hum. Knowing.

[45]  Catherine E. Tucker,et al.  Algorithmic Bias? An Empirical Study of Apparent Gender-Based Discrimination in the Display of STEM Career Ads , 2019, Manag. Sci..

[46]  Alicia Juarrero,et al.  Reviews-Dynamics in Action: Intentional Behavior as a Complex System , 1999 .

[47]  Jahna Otterbacher,et al.  To "See" is to Stereotype , 2021, Proc. ACM Hum. Comput. Interact..