Bias and unfairness in machine learning models: a systematic literature review

One of the difficulties of artificial intelligence is to ensure that model decisions are fair and free of bias. In research, datasets, metrics, techniques, and tools are applied to detect and mitigate algorithmic unfairness and bias. This study aims to examine existing knowledge on bias and unfairness in Machine Learning models, identifying mitigation methods, fairness metrics, and supporting tools. A Systematic Literature Review found 40 eligible articles published between 2017 and 2022 in the Scopus, IEEE Xplore, Web of Science, and Google Scholar knowledge bases. The results show numerous bias and unfairness detection and mitigation approaches for ML technologies, with clearly defined metrics in the literature, and varied metrics can be highlighted. We recommend further research to define the techniques and metrics that should be employed in each case to standardize and ensure the impartiality of the machine learning model, thus, allowing the most appropriate metric to detect bias and unfairness in a given context.

[1]  S. Zuvekas,et al.  Effects of Medicaid Expansion on Insurance Coverage and Health Services Use among Adults with Disabilities Newly Eligible for Medicaid. , 2022, Health services research.

[2]  T. D. Noia,et al.  Recommender systems under European AI regulations , 2022, Commun. ACM.

[3]  Xusheng Du,et al.  Fair Outlier Detection Based on Adversarial Representation Learning , 2022, Symmetry.

[4]  E. Shmueli,et al.  A Review on Fairness in Machine Learning , 2022, ACM Comput. Surv..

[5]  Sidney K. D'Mello,et al.  Integrating Psychometrics and Computing Perspectives on Bias and Fairness in Affective Computing: A case study of automated video interviews , 2021, IEEE Signal Processing Magazine.

[6]  Tai Le Quy,et al.  A survey on datasets for fairness‐aware machine learning , 2021, WIREs Data Mining Knowl. Discov..

[7]  Christian Haas,et al.  Fairness metrics and bias mitigation strategies for rating predictions , 2021, Inf. Process. Manag..

[8]  Alexandros G. Dimakis,et al.  Fairness for Image Generation with Uncertain Sensitive Attributes , 2021, ICML.

[9]  Xia Hu,et al.  Fairness via Representation Neutralization , 2021, NeurIPS.

[10]  Nima Kordzadeh,et al.  Algorithmic bias: review, synthesis, and future research directions , 2021, Eur. J. Inf. Syst..

[11]  M. S. Lee,et al.  Risk Identification Questionnaire for Detecting Unintended Bias in the Machine Learning Development Lifecycle , 2021, AIES.

[12]  Xiaoqian Wang,et al.  Constructing a Fair Classifier with Generated Fair Data , 2021, AAAI.

[13]  Maja Pantic,et al.  Mitigating Demographic Bias in Facial Datasets with Style-Based Multi-attribute Transfer , 2021, International Journal of Computer Vision.

[14]  Christoph Lofi,et al.  Managing bias and unfairness in data for decision support: a survey of machine learning and data engineering approaches to identify and mitigate bias and unfairness within data management and analytics systems , 2021, The VLDB Journal.

[15]  Shira Mitchell,et al.  Algorithmic Fairness: Choices, Assumptions, and Definitions , 2021, Annual Review of Statistics and Its Application.

[16]  Sergio Gago Masagué,et al.  Using Machine Learning in Admissions: Reducing Human and Algorithmic Bias in the Selection Process , 2021, SIGCSE.

[17]  Stefan Lessmann,et al.  Fairness in Credit Scoring: Assessment, Implementation and Profit Implications , 2021, Eur. J. Oper. Res..

[18]  Chen Jinyin,et al.  Fairness Research on Deep Learning , 2021 .

[19]  Pascal D. König,et al.  When Politicization Stops Algorithms in Criminal Justice , 2021 .

[20]  Hong Zhao,et al.  Cost-sensitive hierarchical classification via multi-scale information entropy for data with an imbalanced distribution , 2021, Applied Intelligence.

[21]  Zhongchao Shi,et al.  Algorithm Bias Detection and Mitigation in Lenovo Face Recognition Engine , 2020, NLPCC.

[22]  E. Mayo-Wilson,et al.  The PRISMA 2020 statement: an updated guideline for reporting systematic reviews , 2020, BMJ.

[23]  Olfa Nasraoui,et al.  Evolution and impact of bias in human and machine learning algorithm interaction , 2020, PloS one.

[24]  Tolga Bolukbasi,et al.  The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models , 2020, EMNLP.

[25]  H. V. Jagadish,et al.  Responsible data management , 2020, Proc. VLDB Endow..

[26]  Milija Suknović,et al.  Enforcing fairness in logistic regression algorithm , 2020, 2020 International Conference on INnovations in Intelligent SysTems and Applications (INISTA).

[27]  Annunziata Paviglianiti,et al.  VITAL-ECG: a de-bias algorithm embedded in a gender-immune device , 2020, 2020 IEEE International Workshop on Metrology for Industry 4.0 & IoT.

[28]  Hanna M. Wallach,et al.  Fairlearn: A toolkit for assessing and improving fairness in AI , 2020 .

[29]  J. Bauer,et al.  Harnessing artificial intelligence (AI) to increase wellbeing for all: The case for a new technology diplomacy , 2020, Telecommunications Policy.

[30]  Roberto Esposito,et al.  Constraining deep representations with a noise module for fair classification , 2020, SAC.

[31]  Jason Yon,et al.  Characterising the Digital Twin: A systematic literature review , 2020, CIRP Journal of Manufacturing Science and Technology.

[32]  S. Kraus,et al.  The art of crafting a systematic literature review in entrepreneurship research , 2020, International Entrepreneurship and Management Journal.

[33]  Celia Cintas,et al.  Analyzing Bias in Sensitive Personal Information Used to Train Financial Models , 2019, ArXiv.

[34]  Sylvain Lamprier,et al.  Fair Adversarial Gradient Tree Boosting , 2019, 2019 IEEE International Conference on Data Mining (ICDM).

[35]  Octavio Loyola-González,et al.  Black-Box vs. White-Box: Understanding Their Advantages and Weaknesses From a Practical Point of View , 2019, IEEE Access.

[36]  Yogesh Kumar Dwivedi,et al.  Artificial Intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy , 2019, International Journal of Information Management.

[37]  Kristina Lerman,et al.  A Survey on Bias and Fairness in Machine Learning , 2019, ACM Comput. Surv..

[38]  Xia Hu,et al.  Fairness in Deep Learning: A Computational Perspective , 2019, IEEE Intelligent Systems.

[39]  Eliza M. Grames,et al.  An automated approach to identifying search terms for systematic reviews using keyword co‐occurrence networks , 2019, Methods in Ecology and Evolution.

[40]  Ankur Taly,et al.  Explainable AI in Industry , 2019, KDD.

[41]  Zoubin Ghahramani,et al.  One-Network Adversarial Fairness , 2019, AAAI.

[42]  Martin Wattenberg,et al.  The What-If Tool: Interactive Probing of Machine Learning Models , 2019, IEEE Transactions on Visualization and Computer Graphics.

[43]  Jamil Ammar,et al.  Cyber Gremlin: Social Networking, Machine Learning, and the Global War on Al-Qaida–And Is-Inspired Terrorism , 2019, Int. J. Law Inf. Technol..

[44]  Yunfeng Zhang,et al.  AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias , 2019, IBM Journal of Research and Development.

[45]  Manfred Huber,et al.  Singular race models: addressing bias and accuracy in predicting prisoner recidivism , 2019, PETRA.

[46]  Roberto Morales-Ortega,et al.  Obesity Level Estimation Software based on Decision Trees , 2019, Journal of Computer Science.

[47]  Huzefa Rangwala,et al.  Multi-Differential Fairness Auditor for Black Box Classifiers , 2019, IJCAI.

[48]  Benjamin Paaßen,et al.  Dynamic fairness - Breaking vicious cycles in automatic decision making , 2019, ESANN.

[49]  Harini Suresh,et al.  A Framework for Understanding Unintended Consequences of Machine Learning , 2019, ArXiv.

[50]  Danna Gurari,et al.  Dataset bias: A case study for visual question answering , 2019, ASIST.

[51]  Rayid Ghani,et al.  Aequitas: A Bias and Fairness Audit Toolkit , 2018, ArXiv.

[52]  Sébastien Gambs,et al.  Privacy and Ethical Challenges in Big Data , 2018, FPS.

[53]  Alexandra Chouldechova,et al.  The Frontiers of Fairness in Machine Learning , 2018, ArXiv.

[54]  Inioluwa Deborah Raji,et al.  Model Cards for Model Reporting , 2018, FAT.

[55]  Silvia Chiappa,et al.  A Causal Bayesian Networks Viewpoint on Fairness , 2018, Privacy and Identity Management.

[56]  Julia Rubin,et al.  Fairness Definitions Explained , 2018, 2018 IEEE/ACM International Workshop on Software Fairness (FairWare).

[57]  John Ahmet Erkoyuncu,et al.  A systematic review of augmented reality applications in maintenance , 2018 .

[58]  Novi Quadrianto,et al.  Recycling Privileged Learning and Distribution Matching for Fairness , 2017, NIPS.

[59]  Massimo Aria,et al.  bibliometrix: An R-tool for comprehensive science mapping analysis , 2017, J. Informetrics.

[60]  A. Booth.  Searching for qualitative research for inclusion in systematic reviews: a structured methodological review , 2016, Systematic Reviews.

[61]  A. Gorban,et al.  The Five Factor Model of personality and evaluation of drug consumption risk , 2015, 1506.06297.

[62]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[63]  Jackie MacDonald,et al.  Systematic Approaches to a Successful Literature Review , 2014 .

[64]  I-Cheng Yeh,et al.  The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients , 2009, Expert Syst. Appl..

[65]  Paulo Cortez,et al.  Using data mining to predict secondary school student performance , 2008 .

[66]  S. Cohen.  Medical Expenditure Panel Survey (MEPS) , 2005 .

[67]  F. Giannotti,et al.  Monitoring Fairness in HOLDA , 2022, HHAI.

[68]  Erez Shmueli,et al.  Improving fairness of artificial intelligence algorithms in Privileged-Group Selection Bias data settings , 2021, Expert Syst. Appl..

[69]  SpurlockScott,et al.  Improving machine learning fairness with sampling and adversarial learning , 2021 .

[70]  S. Houde,et al.  Benchmarking Bias Mitigation Algorithms in Representation Learning through Fairness Metrics , 2020 .

[71]  Nicholas Mattei,et al.  We Need Fairness and Explainability in Algorithmic Hiring , 2020, AAMAS.

[72]  Sebastian Schelter,et al.  Fairness-Aware Instrumentation of Preprocessing~Pipelines for Machine Learning , 2020 .

[73]  R. Cardell-Oliver,et al.  Dataset , 2019, Proceedings of the 2nd Workshop on Data Acquisition To Analysis - DATA'19.

[74]  W. Seymour.  Detecting Bias : Does an Algorithm Have to Be Transparent in Order to Be Fair ? , 2018 .

[75]  M. Bacelar.  bias and in A , 2022 .