Reinforcement learning in extensive form games with incomplete information: the bargaining case study

We consider the problem of playing in repeated extensive form games where agents do not have any prior. In this situation classic game theoretical tools are inapplicable and it is common the resort to learning techniques. In this paper, we present a novel learning principle that aims at avoiding oscillations in the agents' strategies induced by the presence of concurrent learners. We apply our algorithm in bargaining, and we experimentally evaluate it showing that using this principle reinforcement learning algorithms can improve their convergence time.

[1]  Nicholas R. Jennings,et al.  Negotiation decision functions for autonomous agents , 1998, Robotics Auton. Syst..

[2]  Nicholas R. Jennings,et al.  An adaptive bilateral negotiation model for e-commerce settings , 2005, Seventh IEEE International Conference on E-Commerce Technology (CEC'05).

[3]  Stefan Napel,et al.  Bilateral Bargaining - Theory and Applications , 2002, Lecture notes in economics and mathematical systems.

[4]  Manuela M. Veloso,et al.  Multiagent learning using a variable learning rate , 2002, Artif. Intell..

[5]  Drew Fudenberg,et al.  Game theory (3. pr.) , 1991 .

[6]  Katia P. Sycara,et al.  Multi-agent learning in extensive games with complete information , 2003, AAMAS '03.

[7]  A. Rubinstein Perfect Equilibrium in a Bargaining Model , 1982 .

[8]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[9]  Ken Binmore,et al.  Learning to be imperfect: The ultimatum game , 1995 .

[10]  Michael L. Littman,et al.  Friend-or-Foe Q-learning in General-Sum Games , 2001, ICML.

[11]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[12]  Keith B. Hall,et al.  Correlated Q-Learning , 2003, ICML.

[13]  Ken Binmore,et al.  Evolutionary Stability in Alternating-Offers Bargaining Games , 1998 .

[14]  David M. Kreps,et al.  Sequential Equilibria Author ( s ) : , 1982 .

[15]  Roger B. Myerson,et al.  Game theory - Analysis of Conflict , 1991 .

[16]  R. Selten,et al.  A Generalized Nash Solution for Two-Person Bargaining Games with Incomplete Information , 1972 .