Issues in rational planning in multi-agent settings

We adopt the decision-theoretic principle of expected utility maximization as a paradigm for designing autonomous rational agents operating in multi-agent environments. We use the formalism of partially observable Markov decision processes and generalize it to include the presence of other agents. Under common assumptions, belief-state MDP can be defined using agents' beliefs that include the agent's knowledge about the environment and about the other agents, including their knowledge about others' states of knowledge. The resulting solution corresponds to what has been called the decision-theoretic approach to game theory. Our approach complements the more traditional game-theoretic approach based on equilibria. Equilibria may be non-unique and do not capture off-equilibrium behaviors. Our approach seeks to avoid these problems, but does so at the cost of having to represent, process and continually update the complex nested state of agent's knowledge.

[1]  John Geanakoplos,et al.  Common Knowledge , 1992, TARK.

[2]  Ken Binmore,et al.  Rationality in the Centipede , 1994, TARK.

[3]  Patrick D. Larkey,et al.  Subjective Probability and the Theory of Games , 1982 .

[4]  Kenneth G. Binmore,et al.  Essays on Foundations of Game Theory , 1982 .

[5]  Drew Fudenberg,et al.  Game theory (3. pr.) , 1991 .

[6]  R. K. Shyamasundar,et al.  Introduction to algorithms , 1996 .

[7]  Joseph Y. Halpern.  Reasoning about only Knowing with Many Agents , 1993, AAAI.

[8]  I. Coles.  Decision Analysis For An Experimental Robot With Unreliable Sensors , 1975, IJCAI.

[9]  Tommy Chin-Chiu Tan,et al.  A Guide to Knowledge and Games , 1988, TARK.

[10]  Adam Brandenburger,et al.  Knowledge and Equilibrium in Games , 1992 .

[11]  R. Aumann,et al.  Epistemic Conditions for Nash Equilibrium , 1995 .

[12]  D. E. Matthews.  Evolution and the Theory of Games , 1977 .

[13]  J. Kagel,et al.  Handbook of Experimental Economics , 1997 .

[14]  Piotr J. Gmytrasiewicz,et al.  Learning models of other agents using influence diagrams , 1999 .

[15]  Joseph Y. Halpern,et al.  Knowledge and common knowledge in a distributed environment , 1984, JACM.

[16]  Daphne Koller,et al.  Multi-Agent Influence Diagrams for Representing and Solving Games , 2001, IJCAI.

[17]  R. Rob,et al.  Learning, Mutation, and Long Run Equilibria in Games , 1993 .

[18]  Ronald Fagin,et al.  A model-theoretic analysis of knowledge , 1991, JACM.

[19]  Sanguk Noh,et al.  Bayesian Update of Recursive Agent Models , 2004, User Modeling and User-Adapted Interaction.

[20]  Edmund H. Durfee,et al.  Rational Coordination in Multi-Agent Environments , 2000, Autonomous Agents and Multi-Agent Systems.

[21]  Polly S Nichols,et al.  Agreeing to disagree. , 2005, General dentistry.

[22]  Milos Hauskrecht,et al.  Value-Function Approximations for Partially Observable Markov Decision Processes , 2000, J. Artif. Intell. Res..

[23]  Richard E. Neapolitan,et al.  Probabilistic reasoning in expert systems - theory and algorithms , 2012 .

[24]  Ronald Fagin,et al.  Reasoning about knowledge , 1995 .

[25]  Jerome A. Feldman,et al.  Decision Theory and Artificial Intelligence II: The Hungry Monkey , 1977, Cogn. Sci..

[26]  Michihiro Kandori,et al.  Evolution of Equilibria in the Long Run: A General Theory and Applications , 1995 .

[27]  Philip J. Reny.  Extensive Games and Common Knowledge , 1988, TARK.

[28]  Colin Camerer.  Individual Decision Making , 2020, The Handbook of Experimental Economics.

[29]  S. Zamir,et al.  Formulation of Bayesian analysis for games with incomplete information , 1985 .

[30]  Peter C. Cheeseman,et al.  In Defense of Probability , 1985, IJCAI.

[31]  Gerhard Lakemeyer.  All They Know About , 1993, AAAI.

[32]  Henk Sol,et al.  Proceedings of the 54th Hawaii International Conference on System Sciences , 1997, HICSS 2015.

[33]  Edmund H. Durfee,et al.  A Logic of Knowledge and Belief for Recursive Modeling: A Preliminary Report , 1992, AAAI.

[34]  John C. Harsanyi,et al.  Games with Incomplete Information Played by "Bayesian" Players, I-III: Part I. The Basic Model& , 2004, Manag. Sci..

[35]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[36]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[37]  Nils J. Nilsson,et al.  Problem-solving methods in artificial intelligence , 1971, McGraw-Hill computer science series.

[38]  Walter Jacobs,et al.  Robot Decisions Based on Maximizing Utility , 1973, IJCAI.

[39]  Craig Boutilier,et al.  Sequential Optimality and Coordination in Multiagent Systems , 1999, IJCAI.

[40]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[41]  Roger B. Myerson,et al.  Game theory - Analysis of Conflict , 1991 .

[42]  Robert J. Aumann,et al.  Interactive epistemology I: Knowledge , 1999, Int. J. Game Theory.

[43]  Craig Boutilier,et al.  Decision-Theoretic Planning: Structural Assumptions and Computational Leverage , 1999, J. Artif. Intell. Res..

[44]  Jon Doyle,et al.  Doyle See Infer Choose Do Perceive Act , 2009 .

[45]  Robert J. Aumann,et al.  Interactive epistemology II: Probability , 1999, Int. J. Game Theory.