Learning tractable probabilistic models for moral responsibility and blame

Moral responsibility is a major concern in autonomous systems, with applications ranging from self-driving cars to kidney exchanges. Although there have been recent attempts to formalise responsibility and blame, among similar notions, the problem of learning within these formalisms has been unaddressed. From the viewpoint of such systems, the urgent questions are: (a) How can models of moral scenarios and blameworthiness be extracted and learnt automatically from data? (b) How can judgements be computed effectively and efficiently, given the split-second decision points faced by some systems? By building on constrained tractable probabilistic learning, we propose and implement a hybrid (between data-driven and rule-based methods) learning framework for inducing models of such scenarios automatically from data and reasoning tractably from them.We report on experiments that compare our system with human judgement in three illustrative domains: lung cancer staging, teamwork management, and trolley problems.

[1]  Peter Singer,et al.  Ethics and Intuitions , 2005, Philosophy after Darwin.

[2]  Toniann Pitassi,et al.  Solving #SAT and Bayesian Inference with Backtracking Search , 2014, J. Artif. Intell. Res..

[3]  J. Thomson The Trolley Problem , 1985 .

[4]  P. Asaro On banning autonomous weapon systems: human rights, automation, and the dehumanization of lethal decision-making , 2012, International Review of the Red Cross.

[5]  James H. Moor,et al.  The Nature, Importance, and Difficulty of Machine Ethics , 2006, IEEE Intelligent Systems.

[6]  Vaishak Belle,et al.  Interventions and Counterfactuals in Tractable Probabilistic Models: Limitations of Contemporary Transformations , 2019, ArXiv.

[7]  Chunyan Miao,et al.  A dataset of human decision-making in teamwork management , 2017, Scientific Data.

[8]  Constantinos Daskalakis,et al.  Learning and Testing Causal Models with Interventions , 2018, NeurIPS.

[9]  Luc De Raedt,et al.  DTProbLog: A Decision-Theoretic Probabilistic Prolog , 2010, AAAI.

[10]  Joseph Y. Halpern,et al.  Causes and explanations: A structural-model approach , 2000 .

[11]  Bertram Gawronski,et al.  Deontological and utilitarian inclinations in moral decision making: a process dissociation approach. , 2013, Journal of personality and social psychology.

[12]  GetoorLise,et al.  Hinge-loss Markov random fields and probabilistic soft logic , 2017 .

[13]  R. McKelvey,et al.  Quantal Response Equilibria for Normal Form Games , 1995 .

[14]  Pascal Poupart,et al.  Sum-Product-Max Networks for Tractable Decision Making: (Extended Abstract) , 2016, AAMAS.

[15]  D K Owens,et al.  Use of Influence Diagrams to Structure Medical Decisions , 1997, Medical decision making : an international journal of the Society for Medical Decision Making.

[16]  Wenji Mao,et al.  Modeling Social Causality and Responsibility Judgment in Multi-Agent Interactions: Extended Abstract , 2012, IJCAI.

[17]  J. Díaz-Herrera Graphical Models for Probabilistic and Causal Reasoning , 2013 .

[18]  Adnan Darwiche,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence SDD: A New Canonical Representation of Propositional Knowledge Bases , 2022 .

[19]  Ilya Volkovich,et al.  A Guide to Learning Arithmetic Circuits , 2016, COLT.

[20]  J. Tenenbaum,et al.  Learning a commonsense moral theory , 2017, Cognition.

[21]  C. Allen,et al.  Artificial Morality: Top-down, Bottom-up, and Hybrid Approaches , 2005, Ethics and Information Technology.

[22]  Vincent Conitzer,et al.  Moral Decision Making Frameworks for Artificial Intelligence , 2017, ISAIM.

[23]  Iyad Rahwan,et al.  A Computational Model of Commonsense Moral Decision Making , 2018, AIES.

[24]  Joseph Y. Halpern,et al.  Causes and Explanations: A Structural-Model Approach. Part I: Causes , 2000, The British Journal for the Philosophy of Science.

[25]  Michael Anderson,et al.  GenEth: a general ethical dilemma analyzer , 2014, AAAI.

[26]  Joseph Y. Halpern,et al.  Responsibility and Blame: A Structural-Model Approach , 2003, IJCAI.

[27]  Wenji Mao,et al.  Automating After Action Review: Attributing Blame or Credit in Team Training , 2003 .

[28]  Pedro M. Domingos,et al.  Sum-product networks: A new deep architecture , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[29]  Chris L. Baker,et al.  Action understanding as inverse planning , 2009, Cognition.

[30]  Michael L. Littman,et al.  Reinforcement Learning as a Framework for Ethical Decision Making , 2016, AAAI Workshop: AI, Ethics, and Society.

[31]  Pamela J. Hinds,et al.  Who Should I Blame? Effects of Autonomy and Transparency on Attributions in Human-Robot Interaction , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[32]  Alan F. T. Winfield,et al.  An architecture for ethical robots inspired by the simulation theory of cognition , 2018, Cognitive Systems Research.

[33]  Joseph Y. Halpern,et al.  Towards Formal Definitions of Blameworthiness, Intention, and Moral Responsibility , 2018, AAAI.

[34]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[35]  Robin Cohen,et al.  Towards Provably Moral AI Agents in Bottom-up Learning Frameworks , 2018, AAAI Spring Symposia.

[36]  J. Pearl Causal inference in statistics: An overview , 2009 .

[37]  Guy Van den Broeck,et al.  Probabilistic Sentential Decision Diagrams , 2014, KR.

[38]  Noah D. Goodman,et al.  Learning the Preferences of Ignorant, Inconsistent Agents , 2015, AAAI.

[39]  Bernhard Nebel,et al.  The HERA approach to morally competent robots , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[40]  Francesca Rossi,et al.  Embedding Ethical Principles in Collective Decision Support Systems , 2016, AAAI.

[41]  Bertram F. Malle,et al.  A Theory of Blame , 2014 .

[42]  Thomas D. Nielsen,et al.  Learning a decision maker's utility function from (possibly) inconsistent behavior , 2004, Artif. Intell..

[43]  Michael Fisher,et al.  Formal verification of ethical choices in autonomous systems , 2016, Robotics Auton. Syst..

[44]  Ross D. Shachter,et al.  Evaluating influence diagrams with decision circuits , 2007, UAI.

[45]  Joshua B. Tenenbaum,et al.  Inference of Intention and Permissibility in Moral Decision Making , 2015, CogSci.

[46]  Aaron Steinfeld,et al.  Effects of blame on trust in human robot interaction , 2014, The 23rd IEEE International Symposium on Robot and Human Interactive Communication.

[47]  Kristian Kersting,et al.  Semantics Derived Automatically from Language Corpora Contain Human-like Moral Choices , 2019, AIES.

[48]  Clifford Nass,et al.  Critic, compatriot, or chump?: Responses to robot blame attribution , 2010, 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[49]  Craig Boutilier,et al.  Context-Specific Independence in Bayesian Networks , 1996, UAI.

[50]  Oren Etzioni,et al.  Incorporating Ethics into Artificial Intelligence , 2017, The Journal of Ethics.

[51]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[52]  Ronald C. Arkin,et al.  An Ethical Governor for Constraining Lethal Action in an Autonomous System , 2009 .

[53]  Guy Van den Broeck,et al.  Learning the Structure of Probabilistic Sentential Decision Diagrams , 2017, UAI.

[54]  Guy Van den Broeck,et al.  Tractable Learning for Structured Probability Spaces: A Case Study in Learning Preference Distributions , 2015, IJCAI.

[55]  Matthias Scheutz,et al.  Modeling Blame to Avoid Positive Face Threats in Natural Language Generation , 2014, INLG.

[56]  J. Henrich,et al.  The Moral Machine experiment , 2018, Nature.

[57]  Franz Pernkopf,et al.  On the Latent Variable Interpretation in Sum-Product Networks , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Judea Pearl,et al.  Graphical Models for Probabilistic and Causal Reasoning , 1997, The Computer Science and Engineering Handbook.

[59]  Pedro M. Domingos,et al.  Learning the Structure of Sum-Product Networks , 2013, ICML.

[60]  Eong Jinkyu,et al.  What is the Trolley Problem , 2015 .