Modeling and Guiding the Creation of Ethical Human-AI Teams

With artificial intelligence continuing to advance, so too do the ethical concerns that can potentially negatively impact humans and the greater society. When these systems begin to interact with humans, these concerns become much more complex and much more important. The field of human-AI teaming provides a relevant example of how AI ethics can have significant and continued effects on humans. This paper reviews research in ethical artificial intelligence, as well as ethical teamwork through the lens of the rapidly advancing field of human-AI teaming, resulting in a model demonstrating the requirements and outcomes of building ethical human-AI teams. The model is created to guide the prioritization of ethics in human-AI teaming by outlining the ethical teaming process, outcomes of ethical teams, and external requirements necessary to ensure ethical human-AI teams. A final discussion is presented on how the developed model will influence the implementation of AI teammates, as well as the development of policy and regulation surrounding the domain in the coming years.

[1]  James A. Roberts,et al.  Generation Y’s Ethical Ideology and Its Potential Workplace Implications , 2013 .

[2]  D. MacInnis,et al.  A Framework for Conceptual Contributions in Marketing , 2011, How to Get Published in the Best Marketing Journals.

[3]  David S. Bright,et al.  Reconsidering Virtue: Differences of Perspective in Virtue Ethics and the Positive Social Sciences , 2014 .

[4]  Matthijs M. Maas,et al.  Regulating for 'Normal AI Accidents': Operational Lessons for the Responsible Governance of Artificial Intelligence Deployment , 2018, AIES.

[5]  Beau G. Schelble,et al.  Human–Autonomy Teaming: A Review and Analysis of the Empirical Literature , 2020, Hum. Factors.

[6]  Jeanna Neefe Matthews,et al.  When Trusted Black Boxes Don't Agree: Incentivizing Iterative Improvement and Accountability in Critical Software Systems , 2020, AIES.

[7]  Seán S. ÓhÉigeartaigh,et al.  An AI Race for Strategic Advantage: Rhetoric and Risks , 2017, AIES.

[8]  Francesca Rossi,et al.  AI4People—An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations , 2018, Minds and Machines.

[9]  Olivier Boissier,et al.  Ethical Judgment of Agents' Behaviors in Multi-Agent Systems , 2016, AAMAS.

[10]  Sandip Sen,et al.  Reputation Based Trust In Human-Agent Teamwork Without Explicit Coordination , 2018, HAI.

[11]  Michael J. Barnes,et al.  The Effects of Level of Autonomy on Human-Agent Teaming for Multi-Robot Control and Local Security Maintenance , 2013 .

[12]  Marija Slavkovik,et al.  Cake, Death, and Trolleys: Dilemmas as benchmarks of ethical decision-making , 2018, AIES.

[13]  Nancy J. Cooke,et al.  Teaming With a Synthetic Teammate: Insights into Human-Autonomy Teaming , 2018, Hum. Factors.

[14]  Carol Taylor.  Ethics in nursing. , 2004, Australian nursing journal.

[15]  Erik Brynjolfsson,et al.  Covid-19 and Remote Work: An Early Look at Us Data , 2020 .

[16]  C. Dianne Martin.  Leadership, teamwork, and ethics in the development of IT professionals , 2007, SGCS.

[17]  L. Roberts,et al.  Becoming a Good Doctor: Perceived Need for Ethics Training Focused on Practical and Professional Development Topics , 2005, Academic psychiatry : the journal of the American Association of Directors of Psychiatric Residency Training and the Association for Academic Psychiatry.

[18]  C. R. D. Oliveira.  Ethics in the operating room; the anesthesiologist's responsibility , 2020 .

[19]  M. Johnstone,et al.  Ethics in Nursing Practice: A Guide to Ethical Decision Making , 2013 .

[20]  Ava Thomas Wright,et al.  Rightful Machines and Dilemmas , 2019, AIES.

[21]  Christopher D. Wickens,et al.  A model for types and levels of human interaction with automation , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[22]  Daniel G. Arce M.,et al.  Working Well with Others: The Evolution of Teamwork and Ethics , 2005 .

[23]  G. Agich.  Joining the Team: Ethics Consultation at the Cleveland Clinic , 2003, HEC forum : an interdisciplinary journal on hospitals' ethical and legal issues.

[24]  Paige Rodeghero,et al.  Invoking Principles of Groupware to Develop and Evaluate Present and Future Human-Agent Teams , 2020, HAI.

[25]  Munindar P. Singh,et al.  Sociotechnical Systems and Ethics in the Large , 2018, AIES.

[26]  N. John Castellan,et al.  Individual and group decision making : current issues , 2013 .

[27]  Subhro Das,et al.  Learning Occupational Task-Shares Dynamics for the Future of Work , 2020, AIES.

[28]  Jessie Y C Chen,et al.  Human–automation interaction for multiple robot control: the effect of varying automation assistance and individual differences on operator performance , 2018, Ergonomics.

[29]  Anna Jobin,et al.  The global landscape of AI ethics guidelines , 2019, Nature Machine Intelligence.

[30]  P. Robinson.  Ethics Training and Development in the Military , 2007, The US Army War College Quarterly: Parameters.

[31]  Ayanna M. Howard,et al.  Emerging challenges in AI and the need for AI ethics education , 2021, AI Ethics.

[32]  Nancy J. Cooke,et al.  Synthetic Teammate Communication and Coordination With Humans , 2015 .

[33]  Ioannis T. Pavlidis,et al.  A Quantitative Perspective on Ethics in Large Team Science , 2014, Sci. Eng. Ethics.

[34]  A. Bleakley.  A common body of care: the ethics and politics of teamwork in the operating theater are inseparable. , 2006, The Journal of medicine and philosophy.

[35]  Marina Jirotka,et al.  Ethical governance is essential to building trust in robotics and artificial intelligence systems , 2018, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[36]  Baobao Zhang,et al.  U.S. Public Opinion on the Governance of Artificial Intelligence , 2019, AIES.

[37]  Luciano Floridi,et al.  From What to How: An Initial Review of Publicly Available AI Ethics Tools, Methods and Research to Translate Principles into Practices , 2019, Science and Engineering Ethics.

[38]  José Hernández-Orallo,et al.  AI Extenders: The Ethical and Societal Implications of Humans Cognitively Extended by AI , 2019, AIES.

[39]  M. Alimoğlu,et al.  Comparison of lecture and team-based learning in medical ethics education , 2019, Nursing ethics.

[40]  Michael A. Rupp,et al.  Intelligent Agent Transparency in Human–Agent Teaming for Multi-UxV Management , 2016, Hum. Factors.

[41]  Sheldon Zedeck,et al.  APA handbook of industrial and organizational psychology, Vol 1: Building and developing the organization. , 2011 .

[42]  A. Killen.  Stories from the Operating Room: moral dilemmas for nurses , 2002, Nursing ethics.

[43]  Eduardo Salas,et al.  Distributed coordination space: Toward a theory of distributed team process and performance , 2003 .

[44]  M. Hall,et al.  The Importance of Trust for Ethics, Law, and Public Policy , 2005, Cambridge Quarterly of Healthcare Ethics.

[45]  Arathi Sethumadhavan,et al.  Trust in Artificial Intelligence , 2018, Ergonomics in Design: The Quarterly of Human Factors Applications.

[46]  Meg Young,et al.  Defining AI in Policy versus Practice , 2019, AIES.

[47]  J. L. Dyer,et al.  Team research and team training: A state-of-the-art review , 1984 .

[48]  Katia P. Sycara,et al.  Influence of cultural factors in dynamic trust in automation , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[49]  David Danks,et al.  Balancing the Benefits of Autonomous Vehicles , 2019, AIES.

[50]  Wilfred W. F. Lau,et al.  Internet ethics of adolescents: Understanding demographic differences , 2014, Comput. Educ..

[51]  Ned Kock,et al.  Trust and leadership in virtual teamwork: A media naturalness perspective , 2004 .

[52]  D. Christakis,et al.  Ethics in a short white coat: the ethical dilemmas that medical students confront , 1993, Academic medicine : journal of the Association of American Medical Colleges.

[53]  Sarah E. Wallace,et al.  Examining the perceived impact of an ethics workshop on interprofessional values and teamwork , 2017, Journal of interprofessional care.

[54]  Daniele Nardi,et al.  Companion Robots: the Hallucinatory Danger of Human-Robot Interactions , 2018, AIES.

[55]  Jette Henderson,et al.  CERTIFAI: A Common Framework to Provide Explanations and Analyse the Fairness and Robustness of Black-box Models , 2020, AIES.

[56]  David Danks,et al.  Regulating Autonomous Vehicles: A Policy Proposal , 2018, AIES.

[57]  Ville Vakkuri,et al.  Implementing AI Ethics in Practice: An Empirical Evaluation of the RESOLVEDD Strategy , 2019, ICSOB.

[58]  M. Laal,et al.  Continuing Education; Lifelong Learning , 2014 .

[59]  Clarence A. Ellis,et al.  Groupware: some issues and experiences , 1991, CACM.

[60]  Nancy J. Cooke,et al.  On Teams, Teamwork, and Team Performance: Discoveries and Developments , 2008, Hum. Factors.

[61]  Roman V. Yampolskiy,et al.  Unethical Research: How to Create a Malevolent Artificial Intelligence , 2016, ArXiv.

[62]  Samuel B. Williams,et al.  ASSOCIATION FOR COMPUTING MACHINERY , 2000 .

[63]  Casey Fiesler,et al.  More Than "If Time Allows": The Role of Ethics in AI Education , 2020, AIES.

[64]  Emanuel Moss,et al.  Owning Ethics: Corporate Logics, Silicon Valley, and the Institutionalization of Ethics , 2019 .

[65]  D. Whetten.  What Constitutes a Theoretical Contribution , 1989 .

[66]  Jessie Y. C. Chen,et al.  Supervisory Control of Robots Using RoboLeader , 2010 .

[67]  Marlies van Steenbergen,et al.  Ethics by Design: Necessity or Curse? , 2018, AIES.

[68]  Ethics and teamwork for pediatric medical imaging procedures: insights from educational play therapy , 2012, Pediatric Radiology.

[69]  Sheldon Zedeck,et al.  Building and developing the organization , 2011 .

[70]  Jessie Y. C. Chen,et al.  Situation awareness-based agent transparency and human-autonomy teaming effectiveness , 2018 .

[71]  Kelly Laas,et al.  What's Next for AI Ethics, Policy, and Governance? A Global Overview , 2020, AIES.

[72]  Gregory Luke Larkin,et al.  Unwitting partners in death-the ethics of teamwork in disaster management. , 2010, The virtual mentor : VM.

[73]  Wei Sun,et al.  How CEO Ethical Leadership Influences Top Management Team Creativity: Evidence From China , 2020, Frontiers in Psychology.

[74]  Stephen Cave,et al.  "Scary Robots": Examining Public Responses to AI , 2019, AIES.

[75]  Francesca Rossi,et al.  Preferences and Ethical Principles in Decision Making , 2018, AAAI Spring Symposia.

[76]  Gareth R. Jones,et al.  The experience and evolution of trust: Implications for cooperation and teamwork , 1998 .

[77]  Graham Sewell,et al.  Doing what comes naturally? Why we need a practical ethics of teamwork , 2005 .

[78]  Jim Tørresen,et al.  A Review of Future and Ethical Perspectives of Robotics and AI , 2018, Front. Robot. AI.

[79]  Sandip Sen,et al.  The Effects of Past Experience on Trust in Repeated Human-Agent Teamwork , 2018, AAMAS.

[80]  Osonde A. Osoba,et al.  Steps Towards Value-Aligned Systems , 2020, AIES.

[81]  Alan F. T. Winfield,et al.  The Dark Side of Ethical Robots , 2016, AIES.

[82]  Allison Woodruff,et al.  Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements , 2019, AIES.

[83]  Francis J. Yammarino,et al.  Team Virtues and Performance: An Examination of Transparency, Behavioral Integrity, and Trust , 2011 .

[84]  Glenn C Graber,et al.  A team-taught interdisciplinary approach to engineering ethics , 2006, Science and engineering ethics.

[85]  W. Hleihel,et al.  Team-based efforts to improve quality of care, the fundamental role of ethics, and the responsibility of health managers: monitoring and management strategies to enhance teamwork. , 2017, Public health.

[86]  Nancy J. Cooke,et al.  Understanding the Role of Trust in Human-Autonomy Teaming , 2019, HICSS.

[87]  Bernd Carsten Stahl,et al.  Ethics and Privacy in AI and Big Data: Implementing Responsible Research and Innovation , 2018, IEEE Security & Privacy.

[88]  Hyrum S. Anderson,et al.  The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation , 2018, ArXiv.

[89]  C. Vandenberghe,et al.  Ethical Leadership and Team Ethical Voice and Citizenship Behavior in the Military: The Roles of Team Moral Efficacy and Ethical Climate , 2020 .

[90]  Beau G. Schelble,et al.  Towards Meaningfully Integrating Human-Autonomy Teaming in Applied Settings , 2020, HAI.

[91]  Deborah Richards,et al.  Do Birds of a Feather Work Better Together? The Impact of Virtual Agent Personality on a Shared Mental Model with Humans during Collaboration , 2015, COOS@AAMAS.

[92]  Daniel F. Jennings,et al.  Ethics and Performance: A Simulation Analysis of Team Decision Making , 1997 .

[93]  E. Salas,et al.  Shared mental models in expert team decision making. , 1993 .

[94]  Alan Frank. Winfield,et al.  Ethical standards in robotics and AI , 2019, Nature Electronics.

[95]  Sami Haddadin,et al.  An embedded ethics approach for AI development , 2020, Nature Machine Intelligence.

[96]  Jeffrey Stamps,et al.  Virtual teams: The new way to work , 1999 .

[97]  Andreas Theodorou,et al.  Towards ethical and socio-legal governance in AI , 2020, Nature Machine Intelligence.

[98]  Jess Whittlestone,et al.  The Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions , 2019, AIES.

[99]  Oren Etzioni,et al.  Incorporating Ethics into Artificial Intelligence , 2017, The Journal of Ethics.

[100]  Matthijs M. Maas,et al.  Should Artificial Intelligence Governance be Centralised?: Design Lessons from History , 2020, AIES.

[101]  Alan P. Sprague,et al.  Tying Ethics to Teamwork Training in a Minimodule , 2019, CSERC.

[102]  A. Arber,et al.  Generosity and the Moral Imagination in the Practice of Teamwork , 2009, Nursing ethics.

[103]  Brent Mittelstadt,et al.  Principles alone cannot guarantee ethical AI , 2019, Nature Machine Intelligence.

[104]  Chunyan Miao,et al.  Building Ethics into Artificial Intelligence , 2018, IJCAI.

[105]  Xiaohui Tao,et al.  Ethics of IT Security Team for Cloud Architecture Infrastructure in Banking and Financial Services Industry , 2019, 2019 IEEE 23rd International Conference on Computer Supported Cooperative Work in Design (CSCWD).

[106]  Ning Wang,et al.  Clustering Behavior to Recognize Subjective Beliefs in Human-Agent Teams , 2018, AAMAS.

[107]  S. K. Horwitz,et al.  The Effects of Team Diversity on Team Outcomes: A Meta-Analytic Review of Team Demography , 2007 .