Analyzing Complex Strategic Interactions in Multi-Agent Systems

We develop a model for analyzing complex games with repeated interactions, for which a full game-theoretic analysis is intractable. Our approach treats exogenously specified, heuristic strategies, rather than the atomic actions, as primitive, and computes a heuristic-payoff table specifying the expected payoffs of the joint heuristic strategy space. We analyze two games based on (i) automated dynamic pricing and (ii) continuous double auction. For each game we compute Nash equilibria of previously published heuristic strategies. To determine the most plausible equilibria, we study the replicator dynamics of a large population playing the strategies. In order to account for errors in estimation of payoffs or improvements in strategies, we also analyze the dynamics and equilibria based on perturbed payoffs.

[1]  V. Smith.  Microeconomic Systems as an Experimental Science , 1982 .

[2]  D. Fudenberg,et al.  Steady state learning and Nash equilibrium , 1993 .

[3]  J. Jordan.  Three Problems in Learning Mixed-Strategy Nash Equilibria , 1993 .

[4]  David M. Kreps,et al.  Learning Mixed Equilibria , 1993 .

[5]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[6]  Jörgen W. Weibull,et al.  Evolutionary Game Theory , 1996 .

[7]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[8]  R. McKelvey,et al.  Computation of equilibria in finite games , 1996 .

[9]  Yoav Freund,et al.  Game theory, on-line prediction and boosting , 1996, COLT '96.

[10]  R. Axelrod.  Evolving New Strategies The Evolution of Strategies in the Iterated Prisoner ' s Dilemma , 1997 .

[11]  D. Cliff.  Minimal-Intelligence Agents for Bargaining Behaviors in Market-Based Environments , 1997 .

[12]  Tilman Börgers,et al.  Learning Through Reinforcement and Replicator Dynamics , 1997 .

[13]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[14]  Jeffrey O. Kephart,et al.  Shopbots and Pricebots , 1999, IJCAI.

[15]  Michael Wooldridge,et al.  Agent-Oriented Software Engineering , 1999, ATAL.

[16]  Jeffrey O. Kephart,et al.  Strategic pricebot dynamics , 1999, EC '99.

[17]  Dean P. Foster,et al.  Regret in the On-Line Decision Problem , 1999 .

[18]  Yishay Mansour,et al.  Nash Convergence of Gradient Dynamics in General-Sum Games , 2000, UAI.

[19]  Paul R. Milgrom,et al.  Putting Auction Theory to Work: The Simultaneous Ascending Auction , 1999, Journal of Political Economy.

[20]  Shou-De Lin,et al.  Designing the Market Game for a Trading Agent Competition , 2001, IEEE Internet Comput..

[21]  Rajarshi Das,et al.  High-performance bidding agents for the continuous double auction , 2001, EC '01.

[22]  John Dickhaut,et al.  Price Formation in Double Auctions , 2001, E-Commerce Agents.

[23]  Peter Stone,et al.  Autonomous Bidding Agents in the Trading Agent Competition , 2001, IEEE Internet Comput..

[24]  Jeffrey O. Kephart,et al.  Probabilistic pricebots , 2001, AGENTS '01.

[25]  Eizo Akiyama,et al.  Chaos in learning a simple two-person game , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[26]  William H. Press,et al.  Numerical recipes in C , 2002 .