Designing Preferences, Beliefs, and Identities for Artificial Intelligence

Research in artificial intelligence, as well as in economics and other related fields, generally proceeds from the premise that each agent has a well-defined identity, well-defined preferences over outcomes, and well-defined beliefs about the world. However, as we design AI systems, we in fact need to specify where the boundaries between one agent and another in the system lie, what objective functions these agents aim to maximize, and to some extent even what belief formation processes they use. The premise of this paper is that as AI is being broadly deployed in the world, we need well-founded theories of, and methodologies and algorithms for, how to design preferences, identities, and beliefs. This paper lays out an approach to address these problems from a rigorous foundation in decision theory, game theory, social choice theory, and the algorithmic and computational aspects of these fields.

[1]  Caspar Hare,et al.  Realism About Tense and Perspective , 2010 .

[2]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[3]  Vincent Conitzer,et al.  Moral Decision Making Frameworks for Artificial Intelligence , 2017, ISAIM.

[4]  Tuomas Sandholm,et al.  Imperfect-Recall Abstractions with Bounds in Games , 2014, EC.

[5]  C. Robert.  Superintelligence: Paths, Dangers, Strategies , 2017 .

[6]  Bo An,et al.  PROTECT - A Deployed Game Theoretic System for Strategic Security Allocation for the United States Coast Guard , 2012, AI Mag..

[7]  Adam Tauman Kalai,et al.  A commitment folk theorem , 2010, Games Econ. Behav..

[8]  Mark Johnston,et al.  On Myself, and Other, Less Important Subjects , 2009 .

[9]  Kevin Waugh,et al.  A Practical Use of Imperfect Recall , 2009, SARA.

[10]  Caspar Oesterheld,et al.  Robust program equilibrium , 2018, Theory and Decision.

[11]  Iyad Rahwan,et al.  A Voting-Based System for Ethical Decision Making , 2017, AAAI.

[12]  Anne Lauscher.  Life 3.0: being human in the age of artificial intelligence , 2019, Internet Histories.

[13]  Milind Tambe,et al.  TRUSTS: Scheduling Randomized Patrols for Fare Inspection in Transit Systems , 2012, IAAI.

[14]  Iyad Rahwan,et al.  The social dilemma of autonomous vehicles , 2015, Science.

[15]  G. Merlo,et al.  Subjectivism and the Mental , 2016 .

[16]  Vincent Conitzer.  A Puzzle about Further Facts , 2018, Erkenntnis.

[17]  Christopher Hitchcock,et al.  Beauty and the Bets , 2004, Synthese.

[18]  Vincent Conitzer,et al.  A Dutch book against sleeping beauties who are evidential decision theorists , 2015, Synthese.

[19]  B. Levinstein,et al.  Cheating Death in Damascus , 2020, The Journal of Philosophy.

[20]  Michael H. Bowling,et al.  No-Regret Learning in Extensive-Form Games with Imperfect Recall , 2012, ICML.

[21]  Caspar Hare,et al.  Self-Bias, Time-Bias, and the Metaphysics of Self and Time , 2007 .

[22]  Troels Bjerre Lund,et al.  Timeability of Extensive-Form Games , 2015, ITCS.

[23]  Anca D. Dragan,et al.  The Off-Switch Game , 2016, IJCAI.

[24]  Vincent Conitzer,et al.  A PAC Framework for Aggregating Agents' Judgments , 2019, AAAI.

[25]  Michael L. Littman,et al.  Reinforcement Learning as a Framework for Ethical Decision Making , 2016, AAAI Workshop: AI, Ethics, and Society.

[26]  Sarit Kraus,et al.  Using Game Theory for Los Angeles Airport Security , 2009, AI Mag..

[27]  Francesca Rossi,et al.  Embedding Ethical Principles in Collective Decision Support Systems , 2016, AAAI.

[28]  Vincent Conitzer,et al.  Computing the optimal strategy to commit to , 2006, EC '06.

[29]  J. J. Valberg.  Dream, Death, and the Self , 2018 .

[30]  Bo An,et al.  Deploying PAWS: Field Optimization of the Protection Assistant for Wildlife Security , 2016, AAAI.

[31]  Joel Pust,et al.  Diachronic Dutch Books and Sleeping Beauty , 2007, Synthese.

[32]  Vincent Conitzer,et al.  Adapting a Kidney Exchange Algorithm to Align with Human Values , 2018, AAAI.

[33]  Adam Elga,et al.  Self‐locating belief and the Sleeping Beauty problem , 2000 .

[34]  Moshe Tennenholtz,et al.  Program equilibrium , 2004, Games Econ. Behav..

[35]  Vincent Conitzer,et al.  Crowdsourcing Societal Tradeoffs , 2015, AAMAS.

[36]  Vincent Conitzer,et al.  Rules for Choosing Societal Tradeoffs , 2016, ISAIM.

[37]  Bernhard von Stengel,et al.  Leadership games with convex strategy sets , 2010, Games Econ. Behav..