”It’s Everybody’s Role to Speak Up... But Not Everyone Will”: Understanding AI Professionals’ Perceptions of Accountability for AI Bias Mitigation

In this paper, we investigate the perceptions of AI professionals for their accountability for mitigating AI bias. Our work is motivated by calls for socially responsible AI development and governance in the face of societal harm but a lack of accountability across the entire socio-technical system. In particular, we explore a gap in the field stemming from the lack of empirical data needed to conclude how real AI professionals view bias mitigation and why individual AI professionals may be prevented from taking accountability even if they have the technical ability to do so. This gap is concerning as larger responsible AI efforts inherently rely on individuals who contribute to designing, developing, and deploying AI technologies and mitigation solutions. Through semi-structured interviews with AI professionals from diverse roles, organizations, and industries working on development projects, we identify that AI professionals are hindered from mitigating AI bias due to challenges that arise from two key areas: (1) their own technical and connotative understanding of AI bias and (2) internal and external organizational factors that inhibit these individuals. In exploring these factors, we reject previous claims that technical aptitude alone prevents accountability for AI bias. Instead, we point to interpersonal and intra-organizational issues that limit agency, empowerment, and overall participation in responsible computing efforts. Furthermore, to support practical approaches to responsible AI, we propose several high-level principled guidelines that will support the understanding, culpability, and mitigation of AI bias and its harm guided by both socio-technical systems and moral disengagement theories.

[1]  Syed Ishtiaque Ahmed,et al.  AI and Disaster Risk: A Practitioner Perspective , 2022, Proc. ACM Hum. Comput. Interact..

[2]  Bhavya Ghai,et al.  D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias , 2022, IEEE Transactions on Visualization and Computer Graphics.

[3]  A. Deshpande,et al.  Responsible AI Systems: Who are the Stakeholders? , 2022, AIES.

[4]  Bianca A. Lepe,et al.  Tech Worker Organizing for Power and Accountability , 2022, FAccT.

[5]  Lewin Schmitt Mapping global AI governance: a nascent regime in a fragmented landscape , 2022 .

[6]  Julian McAuley,et al.  AI-Moderated Decision-Making: Capturing and Balancing Anchoring Bias in Sequential Decision Tasks , 2022, CHI.

[7]  E. Letouzé,et al.  Trust, regulation, and human-in-the-loop AI , 2022, Commun. ACM.

[8]  B. Ramesh,et al.  Establishing Data Provenance for Responsible Artificial Intelligence Systems , 2022, ACM Trans. Manag. Inf. Syst..

[9]  Patrick Mikalef,et al.  Thinking responsibly about responsible AI and ‘the dark side’ of AI , 2022, Eur. J. Inf. Syst..

[10]  D. Kuch,et al.  Economies of Virtue: The Circulation of ‘Ethics’ in Big Tech , 2021, Science as Culture.

[11]  Mihaela Constantinescu,et al.  Understanding responsibility in Responsible AI. Dianoetic virtues and the hard problem of context , 2021, Ethics and Information Technology.

[12]  Rachel Dzombak,et al.  Human-Centered AI , 2021, IEEE Pervasive Comput..

[13]  Lily Hu Tech Ethics: Speaking Ethics to Power, or Power Speaking Ethics? , 2021, J. Soc. Comput..

[14]  Lewin Schmitt Mapping global AI governance: a nascent regime in a fragmented landscape , 2021, AI and Ethics.

[15]  Gordon J. Pace,et al.  Regulating artificial intelligence: a technology regulator's perspective , 2021, ICAIL.

[16]  Will Abramson,et al.  Identifying Roles, Requirements and Responsibilities in Trustworthy AI Systems , 2021, UbiComp/ISWC Adjunct.

[17]  Benson K. Green The Contestation of Tech Ethics: A Sociotechnical Approach to Technology Ethics in Practice , 2021, J. Soc. Comput..

[18]  Dietmar Hübner,et al.  Two Kinds of Discrimination in AI-Based Penal Decision-Making , 2021, SIGKDD Explor..

[19]  G. Mecacci,et al.  Four Responsibility Gaps with Artificial Intelligence: Why they Matter and How to Address them , 2021, Philosophy & Technology.

[20]  Allan Dafoe,et al.  Ethics and Governance of Artificial Intelligence: Evidence from a Survey of Machine Learning Researchers , 2021, J. Artif. Intell. Res..

[21]  Ramya Srinivasan,et al.  Biases in AI Systems , 2021, ACM Queue.

[22]  Tsvi Kuflik,et al.  Educating Software and AI Stakeholders About Algorithmic Fairness, Accountability, Transparency and Ethics , 2021, International Journal of Artificial Intelligence in Education.

[23]  Shira Mitchell,et al.  Algorithmic Fairness: Choices, Assumptions, and Definitions , 2021, Annual Review of Statistics and Its Application.

[24]  Soya Park,et al.  How AI Developers Overcome Communication Challenges in a Multidisciplinary Team , 2021, Proc. ACM Hum. Comput. Interact..

[25]  Kush R. Varshney,et al.  Socially Responsible AI Algorithms: Issues, Purposes, and Challenges , 2021, J. Artif. Intell. Res..

[26]  C. Jones,et al.  Disempowering emotions: The role of educational experiences in social responses to climate change , 2020 .

[27]  M. Wehner,et al.  Discriminated by an algorithm: a systematic review of discrimination and fairness by algorithmic decision-making in the context of HR recruitment and HR development , 2020, Business Research.

[28]  Davide Castelvecchi,et al.  Is facial recognition too biased to be let loose? , 2020, Nature.

[29]  Yan Huang,et al.  Artificial Intelligence and Algorithmic Bias: Source, Detection, Mitigation, and Implications , 2020, Pushing the Boundaries: Frontiers in Impactful OR/OM Research.

[30]  Devansh Saxena,et al.  Collective Organizing and Social Responsibility at CSCW , 2020, CSCW Companion.

[31]  Ray Eitel-Porter,et al.  Beyond the promise: implementing ethical AI , 2020, AI and Ethics.

[32]  Merve Hickok,et al.  Lessons learned from AI ethics principles for future actions , 2020, AI and Ethics.

[33]  Daniel W. Tigard Responsible AI and moral responsibility: a common appreciation , 2020, AI and Ethics.

[34]  A. Howard,et al.  Emerging challenges in AI and the need for AI ethics education , 2020, AI and Ethics.

[35]  Nathanael J. Fast,et al.  When eliminating bias isn’t fair: Algorithmic reductionism and procedural justice in human resource decisions , 2020 .

[36]  T. Hernandez-Boussard,et al.  Bias at warp speed: how AI may contribute to the disparities gap in the time of COVID-19 , 2020, J. Am. Medical Informatics Assoc..

[37]  Arpita Biswas,et al.  The Role of In-Group Bias and Balanced Data: A Comparison of Human and Machine Recidivism Risk Predictions , 2020, COMPASS.

[38]  Onno Bouwmeester,et al.  Reconceptualizing Moral Disengagement as a Process: Transcending Overly Liberal and Overly Conservative Practice in the Field , 2020, Journal of Business Ethics.

[39]  Michael Carl Tschantz,et al.  Human-Centered Approaches to Fair and Responsible AI , 2020, CHI Extended Abstracts.

[40]  Fan Du,et al.  The Impact of Presentation Style on Human-In-The-Loop Detection of Algorithmic Bias , 2020, Graphics Interface.

[41]  Judd Antin,et al.  Introduction to the Special Issue on Negotiating Truth and Trust in Socio-Technical Systems , 2020, ACM Trans. Soc. Comput..

[42]  Steffen Staab,et al.  Bias in data‐driven artificial intelligence systems—An introductory survey , 2020, WIREs Data Mining Knowl. Discov..

[43]  Jenny L. Davis,et al.  Attributions of ethical responsibility by Artificial Intelligence practitioners , 2020 .

[44]  Haydn Belfield,et al.  Activism by the AI Community: Analysing Recent Achievements and Future Prospects , 2020, AIES.

[45]  Matthijs M. Maas,et al.  Should Artificial Intelligence Governance be Centralised?: Design Lessons from History , 2020, AIES.

[46]  Arun Rai,et al.  Explainable AI: from black box to glass box , 2019, Journal of the Academy of Marketing Science.

[47]  Danna Gurari,et al.  Good Systems: Ethical AI for CSCW , 2019, CSCW Companion.

[48]  ForteAndrea,et al.  Reliability and Inter-rater Reliability in Qualitative Research , 2019 .

[49]  Nihar R. Mahapatra,et al.  Ethical Considerations in AI-Based Recruitment , 2019, 2019 IEEE International Symposium on Technology and Society (ISTAS).

[50]  Mark Coeckelbergh,et al.  Artificial Intelligence, Responsibility Attribution, and a Relational Justification of Explainability , 2019, Science and Engineering Ethics.

[51]  Alejandro Barredo Arrieta,et al.  Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI , 2019, Inf. Fusion.

[52]  Anna Jobin,et al.  The global landscape of AI ethics guidelines , 2019, Nature Machine Intelligence.

[53]  Kristina Lerman,et al.  A Survey on Bias and Fairness in Machine Learning , 2019, ACM Comput. Surv..

[54]  Miles Brundage,et al.  The Role of Cooperation in Responsible AI Development , 2019, ArXiv.

[55]  Mai ElSherief,et al.  Mitigating Gender Bias in Natural Language Processing: Literature Review , 2019, ACL.

[56]  Yunfeng Zhang,et al.  Think Your Artificial Intelligence Software Is Fair? Think Again , 2019, IEEE Software.

[57]  Diederick Van Thiel,et al.  Artificial Intelligent Credit Risk Prediction: An Empirical Study of Analytical Artificial Intelligence Tools for Credit Risk Prediction in a Digital Era , 2019, Journal of Accounting and Finance.

[58]  Miroslav Dudík,et al.  Fair Regression: Quantitative Definitions and Reduction-based Algorithms , 2019, ICML.

[59]  Jeanna Neefe Matthews,et al.  Managing Bias in AI , 2019, WWW.

[60]  Miriam C. Buiten Towards Intelligent Regulation of Artificial Intelligence , 2019, European Journal of Risk Regulation.

[61]  Anna‐Maija Lämsä,et al.  (A)moral Agents in Organisations? The Significance of Ethical Organisation Culture for Middle Managers’ Exercise of Moral Agency in Ethical Problems , 2019 .

[62]  Dremliuga Roman,et al.  Artificial Intelligence Legal Policy: Limits of Use of Some Kinds of AI , 2019, ICSCA.

[63]  J. Goldsmith,et al.  Regulating Artificial Intelligence: Proposal for a Global Solution , 2018, AIES.

[64]  Munindar P. Singh,et al.  Sociotechnical Systems and Ethics in the Large , 2018, AIES.

[65]  Miroslav Dudík,et al.  Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need? , 2018, CHI.

[66]  Emily M. Bender,et al.  Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science , 2018, TACL.

[67]  Haiyi Zhu,et al.  The Changing Contours of "Participation" in Data-driven, Algorithmic Ecosystems: Challenges, Tactics, and an Agenda , 2018, CSCW Companion.

[68]  Corinne Cath Governing artificial intelligence: ethical, legal and technical opportunities and challenges , 2018, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[69]  Novi Quadrianto,et al.  Tuning Fairness by Balancing Target Labels , 2018, Frontiers in Artificial Intelligence.

[70]  Rachel K. E. Bellamy,et al.  AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias , 2018, ArXiv.

[71]  Mariarosaria Taddeo,et al.  How AI can be a force for good , 2018, Science.

[72]  Markus Strohmaier,et al.  Homophily influences ranking of minorities in social networks , 2018, Scientific Reports.

[73]  Yiannis Kompatsiaris,et al.  Adaptive Sensitive Reweighting to Mitigate Bias in Fairness-aware Classification , 2018, WWW.

[74]  Hanna M. Wallach,et al.  A Reductions Approach to Fair Classification , 2018, ICML.

[75]  C. Kampf Connecting Corporate and Consumer Social Responsibility Through Social Media Activism , 2018 .

[76]  Virginia Dignum,et al.  Responsible Autonomy , 2017, IJCAI.

[77]  C. Barbaranelli,et al.  Safety Culture, Moral Disengagement, and Accident Underreporting , 2017 .

[78]  Lu Zhang,et al.  A Causal Framework for Discovering and Removing Direct and Indirect Discrimination , 2016, IJCAI.

[79]  Emanuel Gomes,et al.  How communication approaches impact mergers and acquisitions outcomes , 2016 .

[80]  Nick Obradovich,et al.  Collective responsibility amplifies mitigation behaviors , 2016, Climatic Change.

[81]  Christopher D. Wickens,et al.  Complacency and Automation Bias in the Use of Imperfect Automation , 2015, Hum. Factors.

[82]  France Bélanger,et al.  A multi-level socio-technical systems telecommuting framework , 2013 .

[83]  Toniann Pitassi,et al.  Learning Fair Representations , 2013, ICML.

[84]  Z. Xue Preface , 2011 .

[85]  C. Hood The Blame Game: Spin, Bureaucracy, and Self-Preservation in Government , 2010 .

[86]  William A. Wallace,et al.  Value Conflicts in Computational Modeling , 2010, Computer.

[87]  Raja Parasuraman,et al.  Complacency and Bias in Human Use of Automation: An Attentional Integration , 2010, Hum. Factors.

[88]  M. Kane Validity and fairness , 2010 .

[89]  Katherine L. Milkman,et al.  How Can Decision Making Be Improved? , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[90]  Toon Calders,et al.  Classifying without discriminating , 2009, 2009 2nd International Conference on Computer, Control and Communication.

[91]  Johnny Saldaña,et al.  The Coding Manual for Qualitative Researchers , 2009 .

[92]  Ng-Loy Wee Loon Singapore , 2008, Intellectual Property in Asia.

[93]  Guy H. Walker,et al.  A review of sociotechnical systems theory: a classic concept for new command and control paradigms , 2008 .

[94]  Lorrie Faith Cranor,et al.  A Framework for Reasoning About the Human in the Loop , 2008, UPSEC.

[95]  S. Brammer,et al.  Corporate Reputation and Philanthropy: An Empirical Analysis , 2005 .

[96]  Neil Gunningham,et al.  Explaining Corporate Environmental Performance: How Does Regulation Matter? , 2003 .

[97]  G. Ropohl PHILOSOPHY OF SOCIO-TECHNICAL SYSTEMS , 1999 .

[98]  A. Bandura Moral Disengagement in the Perpetration of Inhumanities , 1999, Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc.

[99]  A. Bandura Social cognitive theory: an agentic perspective. , 1999, Annual review of psychology.

[100]  Rob Kling,et al.  Human centered systems in the perspective of organizational and social informatics , 1998, CSOC.

[101]  A. Bandura,et al.  Mechanisms of Moral Disengagement in the Exercise of Moral Agency , 1996 .

[102]  R. Schifter White House , 1996 .

[103]  Albert Bandura,et al.  Selective Activation and Disengagement of Moral Control , 1990 .

[104]  E. Trist,et al.  Some Social and Psychological Consequences of the Longwall Method of Coal-Getting , 1951 .

[105]  J. Krogstie,et al.  From Responsible AI Governance to Competitive Performance: The Mediating Role of Knowledge Management Capabilities , 2022, I3E.

[106]  Zahoor Ul Islam,et al.  Software Engineering Methods for Responsible Artificial Intelligence , 2021, AAMAS.

[107]  Virginia Dignum,et al.  Ensuring Responsible AI in Practice , 2019, Responsible Artificial Intelligence.

[108]  Safinah Ali,et al.  Constructionism, Ethics, and Creativity: Developing Primary and Middle School Artificial Intelligence Education , 2019 .

[109]  Xiangliang Zhang,et al.  Exploiting reject option in classification for social discrimination control , 2018, Inf. Sci..

[110]  Kush R. Varshney,et al.  Optimized Pre-Processing for Discrimination Prevention , 2017, NIPS.

[111]  Erik Soepenberg,et al.  Lowlands sociotechnical design theory and lean production , 2015 .

[112]  Andrea Bonime-Blanc,et al.  Artificial Intelligence: A Strategic Business And Governance Imperative , 2012 .

[113]  Khalid Nadvi,et al.  Global Value Chains, Local Collective Action and Corporate Social Responsibility: a Review of Empirical Evidence , 2010 .

[114]  Adam Kowol The theory of cognitive dissonance By , 2008 .

[115]  Juliet M. Corbin,et al.  Basics of Qualitative Research (3rd ed.): Techniques and Procedures for Developing Grounded Theory , 2008 .

[116]  Kathy Charmaz Constructing grounded theory : a practical guide through qualitative analysis , 2006 .

[117]  Paulo Victor Rodrigues de Carvalho,et al.  Ergonomic field studies in a nuclear power plant control room , 2006 .

[118]  Alan Borning,et al.  Value Sensitive Design: Theory and Methods , 2002 .

[119]  A. Bandura Social Foundations of Thought and Action , 1986 .