Learning about other agents in a dynamic multiagent system

We analyze the problem of learning about other agents in a class of dynamic multiagent systems, where performance of the primary agent depends on behavior of the others. We consider an online version of the problem, where agents must learn models of the others in the course of continual interactions. Various levels of recursive models are implemented in a simulated double auction market. Our experiments show learning agents on average outperform non-learning agents who do not use information about others. Among learning agents, those with minimum recursion assumption generally perform better than the agents with more complicated, though often wrong assumptions.

[1]  Edmund H. Durfee,et al.  The moving target function problem in multi-agent learning , 1998, Proceedings International Conference on Multi Agent Systems (Cat. No.98EX160).

[2]  Sandip Sen,et al.  Correlating Internal Parameters and External Performance: Learning Soccer Agents , 1996, ECAI Workshop LDAIS / ICMAS Workshop LIOME.

[3]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[4]  H. Uzawa.  ON THE STABILITY OF EDGEWORTH'S BARTER PROCESS* , 1962 .

[5]  Edmund H. Durfee,et al.  Learning nested agent models in an information economy , 1998, J. Exp. Theor. Artif. Intell..

[6]  Frank Thuijsman,et al.  Optimality and equilibria in stochastic games , 1992 .

[7]  Edmund H. Durfee,et al.  A decision-theoretic approach to coordinating multiagent interactions , 1991, IJCAI 1991.

[8]  R. McAfee,et al.  Auctions and Bidding , 1986 .

[9]  John Rust,et al.  The Double Auction Market , 1989 .

[10]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[11]  Tuomas Sandholm,et al.  On the Gains and Losses of Speculation in Equilibrium Markets , 1997, IJCAI.

[12]  Martin Lauer,et al.  An Algorithm for Distributed Reinforcement Learning in Cooperative Multi-Agent Systems , 2000, ICML.

[13]  John C. Harsanyi,et al.  Общая теория выбора равновесия в играх / A General Theory of Equilibrium Selection in Games , 1989 .

[14]  S. Sunder.  Lower Bounds for Efficiency of Surplus Extraction in Double Auctions , 1992 .

[15]  Steven R. Williams,et al.  The Double Auction Market: Institutions , 1991 .

[16]  Michael P. Wellman,et al.  Experimental Results on Q-Learning for General-Sum Stochastic Games , 2000, ICML.

[17]  J. Filar,et al.  Competitive Markov Decision Processes , 1996 .