Online learning about other agents in a dynamic multiagent system

WC analyze the problem of learning about other agents in a class of dynamic multiagent systems, where performance of the primary agent depends on behavior of the others, We consider an online version of the problem, where agents must learn models of the others in the course of continual interactions. We implement various lovels of recursive model in a simulated double auction market, Our experiments show that performance of an agent can be quite sensitive to its assumptions about the policies of other agents, and (not surprisingly), when there is substantial uncertainty about the love1 of sophistication of other agents, minimizing assumptions might be the best policy.

[1]  H. Uzawa.  ON THE STABILITY OF EDGEWORTH'S BARTER PROCESS* , 1962 .

[2]  R. McAfee,et al.  Auctions and Bidding , 1986 .

[3]  John Rust,et al.  The Double Auction Market , 1989 .

[4]  Michael P. Wellman.  A Market-Oriented Programming Environment and its Application to Distributed Multicommodity Flow Problems , 1993, J. Artif. Intell. Res..

[5]  John Cubbin,et al.  Optimality and Equilibria in Stochastic Games , 1994 .

[6]  David Carmel,et al.  Opponent Modeling in Multi-Agent Systems , 1995, Adaption and Learning in Multi-Agent Systems.

[7]  Edmund H. Durfee,et al.  A Rigorous, Operational Formalization of Recursive Modeling , 1995, ICMAS.

[8]  Ana L. C. Bazzan,et al.  Evolution of Coordination as a Metaphor for Learning in Multi-Agent Systems , 1996, ECAI Workshop LDAIS / ICMAS Workshop LIOME.

[9]  E. Durfee,et al.  The Impact of Nested Agent Models in an Information Economy , 1996 .

[10]  Sandip Sen,et al.  Correlating Internal Parameters and External Performance: Learning Soccer Agents , 1996, ECAI Workshop LDAIS / ICMAS Workshop LIOME.

[11]  Jürgen Schmidhuber,et al.  Multi-Agent Learning with the Success-Story Algorithm , 1996, ECAI Workshop LDAIS / ICMAS Workshop LIOME.

[12]  Junling Hu,et al.  Self-fulfilling Bias in Multiagent Learning , 1996 .

[13]  Tuomas Sandholm,et al.  On the Gains and Losses of Speculation in Equilibrium Markets , 1997, IJCAI.

[14]  Moshe Tennenholtz,et al.  On the Emergence of Social Conventions: Modeling, Analysis, and Simulations , 1997, Artif. Intell..

[15]  Tucker Balch,et al.  Learning Roles: Behavioral Diversity in Robot Teams , 1997 .

[16]  Edmund H. Durfee,et al.  Agents Learning about Agents: A Framework and Analysis , 1997 .

[17]  Gerhard Weiß.  Distributed Artificial Intelligence Meets Machine Learning Learning in Multi-Agent Environments , 1997, Lecture Notes in Computer Science.

[18]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[19]  Martin Lauer,et al.  An Algorithm for Distributed Reinforcement Learning in Cooperative Multi-Agent Systems , 2000, ICML.