On the Applicability of ML Fairness Notions

ML-based predictive systems are increasingly used to support decisions with a critical impact on individuals' lives such as college admission, job hiring, child custody, criminal risk assessment, etc. As a result, fairness emerged as an important requirement to guarantee that predictive systems do not discriminate against specific individuals or entire sub-populations, in particular, minorities. Given the inherent subjectivity of viewing the concept of fairness, several notions of fairness have been introduced in the literature. This paper is a survey of fairness notions that, unlike other surveys in the literature, addresses the question of "which notion of fairness is most suited to a given real-world scenario and why?". Our attempt to answer this question consists in (1) identifying the set of fairness-related characteristics of the real-world scenario at hand, (2) analyzing the behavior of each fairness notion, and then (3) fitting these two elements to recommend the most suitable fairness notion in every specific setup. The results are summarized in a decision diagram that can be used by practitioners and policy makers to navigate the relatively large catalogue of fairness notions.

[1]  Julia Rubin,et al.  Fairness Definitions Explained , 2018, 2018 IEEE/ACM International Workshop on Software Fairness (FairWare).

[2]  Aleix M. Martínez,et al.  EmotioNet: An Accurate, Real-Time Algorithm for the Automatic Annotation of a Million Facial Expressions in the Wild , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Jeffrey Bellin.  The Inverse Relationship between the Constitutionality and Effectiveness of New York City 'Stop and Frisk' , 2014 .

[4]  Timnit Gebru,et al.  Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification , 2018, FAT.

[5]  Xi Zhang,et al.  Responses to Critiques on Machine Learning of Criminality Perceptions (Addendum of arXiv:1611.04135) , 2017 .

[6]  Guy N. Rothblum,et al.  Probably Approximately Metric-Fair Learning , 2018, ICML.

[7]  Risto Miikkulainen,et al.  GRADE: Machine Learning Support for Graduate Admissions , 2013, AI Mag..

[8]  Catherine Tucker,et al.  Algorithmic bias? An empirical study into apparent gender-based discrimination in the display of STEM career ads , 2019 .

[9]  Yuriy Brun,et al.  Fairness testing: testing software for discrimination , 2017, ESEC/SIGSOFT FSE.

[10]  Toniann Pitassi,et al.  Fairness through awareness , 2011, ITCS '12.

[11]  F Lefford,et al.  A blot on the profession , 1988, British medical journal.

[12]  Gerhard Friedrich,et al.  Recommender Systems - An Introduction , 2010 .

[13]  Michael Carl Tschantz,et al.  Automated Experiments on Ad Privacy Settings , 2014, Proc. Priv. Enhancing Technol..

[14]  John Daniel,et al.  Rankings and Accountability in Higher Education : Uses and Misuses , 2013 .

[15]  Michael Carl Tschantz,et al.  Automated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discrimination , 2014, ArXiv.

[16]  Sherita Hill Golden,et al.  Race/Ethnic Difference in Diabetes and Diabetic Complications , 2013, Current Diabetes Reports.

[17]  A. B. Adeyemo,et al.  ANALYZING EMPLOYEE ATTRITION USING DECISION TREE ALGORITHMS , 2013 .

[18]  Aref Majdara,et al.  Development and application of a Risk Assessment Tool , 2008, Reliab. Eng. Syst. Saf..

[19]  Kristina Lerman,et al.  A Survey on Bias and Fairness in Machine Learning , 2019, ACM Comput. Surv..

[20]  Virginia E. Eubanks.  Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor , 2018 .

[21]  Indre Zliobaite,et al.  A survey on measuring indirect discrimination in machine learning , 2015, ArXiv.

[22]  Krishna P. Gummadi,et al.  Fairness Constraints: Mechanisms for Fair Classification , 2015, AISTATS.

[23]  Jon M. Kleinberg,et al.  Inherent Trade-Offs in the Fair Determination of Risk Scores , 2016, ITCS.

[24]  Indre Zliobaite,et al.  On the relation between accuracy and fairness in binary classification , 2015, ArXiv.

[25]  Andrew D. Selbst,et al.  Big Data's Disparate Impact , 2016 .

[26]  Brian W. Powers,et al.  Dissecting racial bias in an algorithm used to manage the health of populations , 2019, Science.

[27]  Kristian Lum,et al.  An algorithm for removing sensitive information: Application to race-independent recidivism prediction , 2017, The Annals of Applied Statistics.

[28]  Matt J. Kusner,et al.  Counterfactual Fairness , 2017, NIPS.

[29]  Sharad Goel,et al.  The Problem of Infra-Marginality in Outcome Tests for Discrimination , 2016, 1607.05376.

[30]  Sharad Goel,et al.  The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning , 2018, ArXiv.

[31]  Krishna P. Gummadi,et al.  iFair: Learning Individually Fair Data Representations for Algorithmic Decision Making , 2018, 2019 IEEE 35th International Conference on Data Engineering (ICDE).

[32]  Pratik Gajane,et al.  On formalizing fairness in prediction with machine learning , 2017, ArXiv.

[33]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[34]  Bernhard Schölkopf,et al.  Avoiding Discrimination through Causal Reasoning , 2017, NIPS.

[35]  Guy N. Rothblum,et al.  Fairness Through Computationally-Bounded Awareness , 2018, NeurIPS.

[36]  Alexandra Chouldechova,et al.  Does mitigating ML's impact disparity require treatment disparity? , 2017, NeurIPS.

[37]  Xi Zhang,et al.  Automated Inference on Criminality using Face Images , 2016, ArXiv.

[38]  Luca Oneto,et al.  Fairness in Machine Learning , 2020, INNSBDDL.

[39]  Randall S Sexton,et al.  Employee turnover: a neural network solution , 2005, Comput. Oper. Res..

[40]  M. Kearns,et al.  Fairness in Criminal Justice Risk Assessments: The State of the Art , 2017, Sociological Methods & Research.

[41]  W. Cliby,et al.  Skene's gland adenocarcinoma with increased serum level of prostate-specific antigen. , 1994, Gynecologic oncology.

[42]  Hannah Lebovits.  Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor , 2018, Public Integrity.

[43]  Amir Mohammad Esmaieeli Sikaroudi,et al.  A data mining approach to employee turnover prediction (case study: Arak automotive parts manufacturing) , 2015 .

[44]  Solon Barocas,et al.  Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions , 2018, 1811.07867.

[45]  Nathan Srebro,et al.  Equality of Opportunity in Supervised Learning , 2016, NIPS.

[46]  COMPAS Risk Scales : Demonstrating Accuracy Equity and Predictive Parity Performance of the COMPAS Risk Scales in Broward County , 2016 .

[47]  Carlos Eduardo Scheidegger,et al.  Certifying and Removing Disparate Impact , 2014, KDD.

[48]  Krishna P. Gummadi,et al.  Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment , 2016, WWW.

[49]  Kenneth A. Bollen,et al.  Structural Equations with Latent Variables , 1989 .

[50]  Fred Schmidt,et al.  Predictive Validity of the Structured Assessment for Violence Risk in Youth (SAVRY) With Juvenile Offenders , 2008 .

[51]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[52]  Michael Luca,et al.  Supplemental Appendix for : Productivity and Selection of Human Capital with Machine Learning , 2016 .

[53]  Sandra Wachter,et al.  A Right to Reasonable Inferences: Re-Thinking Data Protection Law in the Age of Big Data and AI , 2018 .

[54]  Nan Jiang,et al.  Children in the public benefit system at risk of maltreatment: identification via predictive modeling. , 2013, American journal of preventive medicine.

[55]  Alexandra Chouldechova,et al.  Fair prediction with disparate impact: A study of bias in recidivism prediction instruments , 2016, Big Data.

[56]  Mark Wilson,et al.  Unfair Treatment? The Case of Freedle, the SAT, and the Standardization Approach to Differential Item Functioning , 2010 .

[57]  K. Crenshaw.  Mapping the margins: intersectionality, identity politics, and violence against women of color , 1991 .

[58]  Harini Suresh,et al.  A Framework for Understanding Unintended Consequences of Machine Learning , 2019, ArXiv.

[59]  Suresh Venkatasubramanian,et al.  On the (im)possibility of fairness , 2016, ArXiv.

[60]  M. Hernán,et al.  Results on differential and dependent measurement error of the exposure and the outcome using signed directed acyclic graphs. , 2012, American journal of epidemiology.

[61]  Thomas S. Woodson.  Weapons of math destruction , 2018, Journal of Responsible Innovation.

[62]  Julie D. Golomb,et al.  A Neural Basis of Facial Action Recognition in Humans , 2016, The Journal of Neuroscience.

[63]  Tony Doyle,et al.  Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy , 2017, Inf. Soc..

[64]  Yue Zhao,et al.  Employee Turnover Prediction with Machine Learning: A Reliable Approach , 2018, IntelliSys.

[65]  Ameet Talwalkar,et al.  FACT: A Diagnostic for Group Fairness Trade-offs , 2020, ICML.

[66]  Suresh Venkatasubramanian,et al.  A comparative study of fairness-enhancing interventions in machine learning , 2018, FAT.

[67]  Ameet Talwalkar,et al.  Model-Agnostic Characterization of Fairness Trade-offs , 2020, ICML 2020.

[68]  J. Pearl,et al.  Causal Inference in Statistics: A Primer , 2016 .

[69]  Avi Feller,et al.  Algorithmic Decision Making and the Cost of Fairness , 2017, KDD.

[70]  Reuben Binns,et al.  On the apparent conflict between individual and group fairness , 2019, FAT*.

[71]  Afshin Dehghan,et al.  DAGER: Deep Age, Gender and Emotion Recognition Using Convolutional Neural Network , 2017, ArXiv.

[72]  Yongjie Yang,et al.  Fairness in Algorithmic Decision-Making: Applications in Multi-Winner Voting, Machine Learning, and Recommender Systems , 2019, Algorithms.

[73]  Faisal Kamiran,et al.  Quantifying explainable discrimination and removing illegal discrimination in automated decision making , 2012, Knowledge and Information Systems.

[74]  Seth Neel,et al.  Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness , 2017, ICML.