How Do Fairness Definitions Fare?: Examining Public Attitudes Towards Algorithmic Definitions of Fairness

What is the best way to define algorithmic fairness? While many definitions of fairness have been proposed in the computer science literature, there is no clear agreement over a particular definition. In this work, we investigate ordinary people's perceptions of three of these fairness definitions. Across two online experiments, we test which definitions people perceive to be the fairest in the context of loan decisions, and whether fairness perceptions change with the addition of sensitive information (i.e., race of the loan applicants). Overall, one definition (calibrated fairness) tends to be more pre- ferred than the others, and the results also provide support for the principle of affirmative action.

[1]  Evelyn Ellis,et al.  EU Anti-Discrimination Law , 2005 .

[2]  Aaron Roth,et al.  Fairness in Learning: Classic and Contextual Bandits , 2016, NIPS.

[3]  Joshua D. Greene.  Moral Tribes: Emotion, Reason, and the Gap Between Us and Them , 2001 .

[4]  Krishna P. Gummadi,et al.  Beyond Distributive Fairness in Algorithmic Decision Making: Feature Selection for Procedurally Fair Learning , 2018, AAAI.

[5]  COMPAS Risk Scales : Demonstrating Accuracy Equity and Predictive Parity Performance of the COMPAS Risk Scales in Broward County , 2016 .

[6]  Jon M. Kleinberg,et al.  Inherent Trade-Offs in the Fair Determination of Risk Scores , 2016, ITCS.

[7]  Iyad Rahwan,et al.  The social dilemma of autonomous vehicles , 2015, Science.

[8]  Nathan Srebro,et al.  Equality of Opportunity in Supervised Learning , 2016, NIPS.

[9]  Jun Zhao,et al.  'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions , 2018, CHI.

[10]  Yang Liu,et al.  Calibrated Fairness in Bandits , 2017, ArXiv.

[11]  Min Kyung Lee,et al.  A Human-Centered Approach to Algorithmic Services: Considerations for Fair and Motivating Smart Community Service Management that Allocates Donations to Non-Profit Organizations , 2017, CHI.

[12]  Alexandra Chouldechova,et al.  Fair prediction with disparate impact: A study of bias in recidivism prediction instruments , 2016, Big Data.

[14]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[15]  J. Henrich,et al.  The Moral Machine experiment , 2018, Nature.

[16]  Pratik Gajane,et al.  On formalizing fairness in prediction with machine learning , 2017, ArXiv.

[17]  M. Yaari,et al.  On dividing justly , 1984 .

[18]  Adams Js.  Towards an understanding of inequity , 1963 .

[19]  Allison Woodruff,et al.  A Qualitative Exploration of Perceptions of Algorithmic Fairness , 2018, CHI.

[20]  Michael Carl Tschantz,et al.  Exploring User Perceptions of Discrimination in Online Targeted Advertising , 2017, USENIX Security Symposium.

[21]  J. S. Adams,et al.  TOWARD AN UNDERSTANDING OF INEQUITY. , 1963, Journal of abnormal psychology.

[22]  Krishna P. Gummadi,et al.  Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Prediction , 2018, WWW.

[23]  R. Bassett.  Moral Tribes: Emotion, Reason, and the Gap between Us and Them , 2015 .

[24]  Min Kyung Lee.  Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management , 2018, Big Data Soc..

[25]  Toniann Pitassi,et al.  Fairness through awareness , 2011, ITCS '12.

[26]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[27]  Min Kyung Lee.  Algorithmic Mediation in Group Decisions: Fairness Perceptions of Algorithmically Mediated vs. Discussion-Based Social Division , 2017, CSCW.

[28]  J. S. Adams,et al.  Inequity In Social Exchange , 1965 .

[29]  Aaron Roth,et al.  Fairness in Reinforcement Learning , 2016, ICML.

[30]  Emma Pierson,et al.  Gender differences in beliefs about algorithmic fairness , 2017, ArXiv.

[31]  Max H. Bazerman,et al.  Perceptions of Fairness in Interpersonal and Individual Choice Situations , 1995 .

[32]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.