On the current and emerging challenges of developing fair and ethical AI solutions in financial services

AI has found a wide range of application areas in the financial services industry. As the number and the criticality of the applications continue to increase, fair and ethical AI has emerged as an industry-wide objective. In recent years, numerous ethical principles, guidelines and techniques have been proposed. However, the model development organizations face serious challenges in building ethical AI solutions. This paper focuses on the overarching issues model development teams face, which range from the design and implementation complexities, to the shortage of tools, and the lack of organizational constructs. It argues that focusing on the practical considerations is an important step in bridging the gap between the high-level ethics principles and the deployed AI applications, as well as starting industry-wide conversations toward solution approaches.

[1]  Brent Mittelstadt,et al.  Principles alone cannot guarantee ethical AI , 2019, Nature Machine Intelligence.

[2]  Daniel M. Bartels,et al.  Moral Judgment and Decision Making , 2014 .

[3]  Andrew D. Selbst,et al.  Big Data's Disparate Impact , 2016 .

[4]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[5]  Nicole Immorlica,et al.  The Disparate Effects of Strategic Manipulation , 2018, FAT.

[6]  Susan Leigh Anderson,et al.  Asimov’s “three laws of robotics” and machine metaethics , 2008, AI & SOCIETY.

[7]  Theodore Rostow.  What Happens When an Acquaintance Buys Your Data?: A New Privacy Harm in the Age of Data Brokers , 2017 .

[8]  Ananthram Swami,et al.  Practical Black-Box Attacks against Machine Learning , 2016, AsiaCCS.

[9]  Kristina Lerman,et al.  A Survey on Bias and Fairness in Machine Learning , 2019, ACM Comput. Surv..

[10]  C. Véliz.  Three things digital ethics can learn from medical ethics , 2019, Nature Electronics.

[11]  Joel Z. Leibo,et al.  Multi-agent Reinforcement Learning in Sequential Social Dilemmas , 2017, AAMAS.

[12]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[13]  J. Prinz.  The Emotional Construction of Morals , 2007 .

[14]  Solon Barocas,et al.  Problem Formulation and Fairness , 2019, FAT.

[15]  Matt J. Kusner,et al.  Counterfactual Fairness , 2017, NIPS.

[16]  Roman V. Yampolskiy.  Unpredictability of AI , 2019, ArXiv.

[17]  Toniann Pitassi,et al.  Learning Adversarially Fair and Transferable Representations , 2018, ICML.

[18]  Kinga Kita,et al.  Google Street View image of a house predicts car accident risk of its resident , 2019, 1904.05270.

[19]  R. Berk.  A Primer on Fairness in Criminal Justice Risk Assessments , 2016 .

[20]  J. Gilmour,et al.  Evolutionary Ethics , 1944, Nature.

[21]  Anca D. Dragan,et al.  The Social Cost of Strategic Classification , 2018, FAT.

[22]  Kevin B. Korb.  Ethics of AI , 2007, Encyclopedia of Information Ethics and Security.

[23]  Danah Boyd,et al.  Fairness and Abstraction in Sociotechnical Systems , 2019, FAT.

[24]  M. Varacallo,et al.  2019 , 2019, Journal of Surgical Orthopaedic Advances.

[25]  J. Henrich,et al.  The Moral Machine experiment , 2018, Nature.

[26]  Stephan Günnemann,et al.  Adversarial Attacks on Graph Neural Networks , 2019, GI-Jahrestagung.

[27]  Ross P. Buckley,et al.  The Dark Side of Digital Financial Transformation: The New Risks of FinTech and the Rise of TechRisk , 2019, SSRN Electronic Journal.

[28]  Imran Gulamhuseinwala,et al.  FinTech is Gaining Traction and Young, High-Income Users are the Early Adopters , 2015 .

[29]  Michael Veale,et al.  Demystifying the Draft EU Artificial Intelligence Act , 2021, ArXiv.

[30]  Anna Jobin,et al.  The global landscape of AI ethics guidelines , 2019, Nature Machine Intelligence.

[31]  J. Gips.  Towards the ethical robot , 1995 .

[32]  Kelly Laas,et al.  What's Next for AI Ethics, Policy, and Governance? A Global Overview , 2020, AIES.

[33]  Roman V. Yampolskiy,et al.  Taxonomy of Pathways to Dangerous AI , 2015, ArXiv.

[34]  Mai ElSherief,et al.  Mitigating Gender Bias in Natural Language Processing: Literature Review , 2019, ACL.

[35]  윤재량.  2004 , 2019, The Winning Cars of the Indianapolis 500.

[36]  Alan F. T. Winfield,et al.  The Dark Side of Ethical Robots , 2016, AIES.

[37]  Xiao-Yang Liu,et al.  Practical Deep Reinforcement Learning Approach for Stock Trading , 2018, ArXiv.

[38]  Alan F. T. Winfield,et al.  An architecture for ethical robots inspired by the simulation theory of cognition , 2018, Cognitive Systems Research.

[39]  Alexandra Chouldechova,et al.  A snapshot of the frontiers of fairness in machine learning , 2020, Commun. ACM.

[40]  Robert L. Shook.  The book of why , 1983 .

[41]  K. Crawford,et al.  Social Media, Financial Algorithms and the Hack Crash , 2015 .

[42]  Cynthia Rudin,et al.  Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead , 2018, Nature Machine Intelligence.

[43]  Susan Grover.  The Business Necessity Defense in Disparate Impact Discrimination Cases , 1996 .

[44]  Alexandra Chouldechova,et al.  Fair prediction with disparate impact: A study of bias in recidivism prediction instruments , 2016, Big Data.

[45]  Richard W. Counts.  Forging the future , 1988, J. Comput. Aided Mol. Des..

[46]  Varun Gupta,et al.  On the Compatibility of Privacy and Fairness , 2019, UMAP.

[47]  Marina Jirotka,et al.  Ethical governance is essential to building trust in robotics and artificial intelligence systems , 2018, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[48]  R. Posner.  The Federal Trade Commission , 1969 .

[49]  Colin Allen,et al.  Prolegomena to any future artificial moral agent , 2000, J. Exp. Theor. Artif. Intell..

[50]  Suresh Venkatasubramanian,et al.  On the (im)possibility of fairness , 2016, ArXiv.

[51]  Colm O'Riordan,et al.  An evolutionary approach to deception in multi-agent systems , 2008, Artificial Intelligence Review.

[52]  Blaine Nelson,et al.  The security of machine learning , 2010, Machine Learning.

[53]  Sharad Goel,et al.  The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning , 2018, ArXiv.

[54]  Jon M. Kleinberg,et al.  Inherent Trade-Offs in the Fair Determination of Risk Scores , 2016, ITCS.

[55]  Kirsten E. Martin.  Ethical Issues in the Big Data Industry , 2015, MIS Q. Executive.

[56]  David Gunning,et al.  DARPA's explainable artificial intelligence (XAI) program , 2019, IUI.

[57]  佐藤誠三郞,et al.  1997 , 1978, The Winning Cars of the Indianapolis 500.

[58]  M. Cannarsa.  Ethics Guidelines for Trustworthy AI , 2021, The Cambridge Handbook of Lawyering in the Digital Age.

[59]  Natalia Criado,et al.  Attesting Biases and Discrimination using Language Semantics , 2019, ArXiv.

[60]  Emily Denton,et al.  Characterising Bias in Compressed Models , 2020, ArXiv.

[61]  David Weinberger,et al.  Accountability of AI Under the Law: The Role of Explanation , 2017, ArXiv.

[62]  Stuart J. Russell.  Rationality and Intelligence , 1995, IJCAI.

[63]  Maaike Cima,et al.  Psychopaths know right from wrong but don't care. , 2010, Social cognitive and affective neuroscience.

[64]  Jeremy Sugarman,et al.  The inescapable relevance of bioethics for the practicing clinician. , 2006, Chest.

[65]  A. Tutt.  An FDA for Algorithms , 2016 .

[66]  A. Hosny,et al.  The Dataset Nutrition Label , 2020, Data Protection and Privacy.

[67]  F. Z. Borgesius,et al.  AFFINITY PROFILING AND DISCRIMINATION BY ASSOCIATION IN ONLINE BEHAVIORAL ADVERTISING , 2021 .

[68]  Udo Schuklenk,et al.  On the ethics of AI ethics. , 2020, Bioethics.

[69]  Manuel Gomez-Rodriguez,et al.  Decisions, Counterfactual Explanations and Strategic Behavior , 2020, NeurIPS.