Reasoning about rationality and beliefs

In order to succeed, agents playing games must reason about the mechanics of the game, the strategies of other agents, other agentsý reasoning about their strategies, and the rationality of agents. This paper presents a compact, natural and highly expressive language for reasoning about the beliefs and rationality of agentsý decision-making processes in games. It extends a previous version of the language in a number of important ways. Agents can reason directly about the rationality of other agents; agentsý beliefs are allowed to conflict with one another, including situations in which these beliefs form a cyclic structure; agentsý play can deviate from the normative game theoretic solution. The paper formalizes the equilibria that holds with respect to agentsý models and behavior, and provides algorithms for computing it. It also shows that the language is strictly more expressive than that of Bayesian games.

[1]  G. Bonanno.  Modal logic and game theory: two alternative approaches , 2002 .

[2]  S. Zamir,et al.  Formulation of Bayesian analysis for games with incomplete information , 1985 .

[3]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[4]  M. Rabin.  Published by: American , 2022 .

[5]  D. Koller,et al.  Efficient Computation of Equilibria for Extensive Two-Person Games , 1996 .

[6]  Ya'akov Gal,et al.  A language for modeling agents' decision making processes in games , 2003, AAMAS '03.

[7]  Darse Billings,et al.  The First International RoShamBo Programming Competition , 2000, J. Int. Comput. Games Assoc..

[8]  J. Nash.  THE BARGAINING PROBLEM , 1950, Classics in Game Theory.

[9]  Ya'akov Gal,et al.  A Language for Opponent Modeling in Repeated Games , 2003 .

[10]  S. Vajda.  Some topics in two-person games , 1971 .

[11]  H. Brachinger,et al.  Decision analysis , 1997 .

[12]  Michael P. Wellman,et al.  Price Prediction Strategies for Market-Based Scheduling , 2004, ICAPS.

[13]  Colin Camerer.  Behavioral Game Theory , 1990 .

[14]  Dan Geiger,et al.  Identifying independence in bayesian networks , 1990, Networks.

[15]  Radford M. Neal.  Connectionist Learning of Belief Networks , 1992, Artif. Intell..

[16]  Gary E. Bolton,et al.  A stress test of fairness measures in models of social utility , 2005 .

[17]  Ya'akov Gal,et al.  Learning Social Preferences in Games , 2004, AAAI.

[18]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[19]  S. Morris.  The Common Prior Assumption in Economic Theory , 1995, Economics and Philosophy.

[20]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[21]  Daphne Koller,et al.  Multi-agent algorithms for solving graphical games , 2002, AAAI/IAAI.

[22]  R. Aumann,et al.  Epistemic Conditions for Nash Equilibrium , 1995 .

[23]  W. Güth,et al.  An experimental analysis of ultimatum bargaining , 1982 .

[24]  Nir Friedman,et al.  The Bayesian Structural EM Algorithm , 1998, UAI.

[25]  T. Schelling.  The Strategy of Conflict , 1963 .

[26]  Ronald A. Howard,et al.  Influence Diagrams , 2005, Decis. Anal..

[27]  Sarit Kraus,et al.  The influence of social dependencies on decision-making: initial investigations with a new game , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[28]  Ronald Fagin,et al.  Reasoning about knowledge , 1995 .

[29]  Michael P. Wellman,et al.  The Michigan Internet AuctionBot: a configurable auction server for human and software agents , 1998, AGENTS '98.

[30]  Giacomo Bonanno,et al.  How to make sense of the common prior assumption under incomplete information , 1999, Int. J. Game Theory.

[31]  Piotr J. Gmytrasiewicz,et al.  Learning models of other agents using influence diagrams , 1999 .

[32]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[33]  Daphne Koller,et al.  Probabilistic reasoning for complex systems , 1999 .

[34]  Michael P. Wellman,et al.  Learning about other agents in a dynamic multiagent system , 2001, Cognitive Systems Research.

[35]  Colin Camerer.  Behavioral Game Theory: Experiments in Strategic Interaction , 2003 .

[36]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[37]  Colin Camerer,et al.  A cognitive hierarchy theory of one-shot games: Some preliminary results , 2003 .

[38]  Frank Jensen,et al.  From Influence Diagrams to junction Trees , 1994, UAI.

[39]  J. Geanakoplos,et al.  Psychological games and sequential rationality , 1989 .

[40]  Martha E. Pollack,et al.  Intelligent Technology for an Aging Population: The Use of AI to Assist Elders with Cognitive Impairment , 2005, AI Mag..

[41]  Michael L. Littman,et al.  Graphical Models for Game Theory , 2001, UAI.

[42]  J. Nash.  Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[43]  Eddie Dekel,et al.  Hierarchies of Beliefs and Common Knowledge , 1993 .

[44]  Rina Dechter,et al.  Bucket Elimination: A Unifying Framework for Reasoning , 1999, Artif. Intell..

[45]  Leslie Pack Kaelbling,et al.  Playing is believing: The role of beliefs in multi-agent learning , 2001, NIPS.

[46]  Clifford Stein,et al.  Introduction to Algorithms, 2nd edition. , 2001 .

[47]  Howard Raiffa,et al.  Games And Decisions , 1958 .

[48]  E. Kalai,et al.  Rational Learning Leads to Nash Equilibrium , 1993 .

[49]  Avi Pfeffer,et al.  Representations and Solutions for Game-Theoretic Problems , 1997, Artif. Intell..

[50]  Peter Stone,et al.  Bidding for customer orders in TAC SCM , 2004, AAMAS'04.

[51]  Richard D. Lawrence.  A Machine-Learning Approach to Optimal Bid Pricing , 2003 .

[52]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[53]  Daphne Koller,et al.  Multi-Agent Influence Diagrams for Representing and Solving Games , 2001, IJCAI.

[54]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[55]  Rajarshi Das,et al.  Agent-Human Interactions in the Continuous Double Auction , 2001, IJCAI.

[56]  Daphne Koller,et al.  A Continuation Method for Nash Equilibria in Structured Games , 2003, IJCAI.

[57]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[58]  L. Thompson,et al.  Social Utility and Decision Making in Interpersonal Contexts , 1989 .

[59]  David Carmel,et al.  Opponent Modeling in Multi-Agent Systems , 1995, Adaption and Learning in Multi-Agent Systems.

[60]  Daphne Koller,et al.  Utilities as Random Variables: Density Estimation and Structure Discovery , 2000, UAI.

[61]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[62]  Jonathan Schaeffer,et al.  Improved Opponent Modeling in Poker , 2000 .

[63]  M. Bazerman.  Judgment in Managerial Decision Making , 1990 .

[64]  M. Rabin,et al.  Understanding Social Preference with Simple Tests , 2001 .

[65]  Daphne Koller,et al.  Learning an Agent's Utility Function by Observing Behavior , 2001, ICML.

[66]  J. Harsanyi.  Games with Incomplete Information Played by 'Bayesian' Players, Part III. The Basic Probability Distribution of the Game , 1968 .

[67]  Ross D. Shachter.  Evaluating Influence Diagrams , 1986, Oper. Res..

[68]  Andrew McLennan,et al.  Gambit: Software Tools for Game Theory , 2006 .

[69]  T. Schelling,et al.  The Strategy of Conflict. , 1961 .

[70]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[71]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .