Communication complexity as a lower bound for learning in games

A fast-growing body of research in the AI and machine learning communities addresses learning in games, where there are multiple learners with different interests. This research adds to more established research on learning in games conducted in economics. In part because of a clash of fields, there are widely varying requirements on learning algorithms in this domain. The goal of this paper is to demonstrate how communication complexity can be used as a lower bound on the required learning time or cost. Because this lower bound does not assume any requirements on the learning algorithm, it is universal, applying under any set of requirements on the learning algorithm.We characterize exactly the communication complexity of various solution concepts from game theory, namely Nash equilibrium, iterated dominant strategies (both strict and weak), and backwards induction. This gives the tighest lower bounds on learning in games that can be obtained with this method.

[1]  Yishay Mansour,et al.  Nash Convergence of Gradient Dynamics in General-Sum Games , 2000, UAI.

[2]  Moshe Tennenholtz,et al.  Local-Effect Games , 2003, IJCAI.

[3]  Michael L. Littman,et al.  Graphical Models for Game Theory , 2001, UAI.

[4]  Xiaofeng Wang,et al.  Reinforcement Learning to Play an Optimal Nash Equilibrium in Team Markov Games , 2002, NIPS.

[5]  Eitan Zemel,et al.  The Complexity of Eliminating Dominated Strategies , 1993, Math. Oper. Res..

[6]  Peter Stone,et al.  A polynomial-time nash equilibrium algorithm for repeated games , 2003, EC '03.

[7]  James Hannan,et al.  4. APPROXIMATION TO RAYES RISK IN REPEATED PLAY , 1958 .

[8]  Christos H. Papadimitriou,et al.  Algorithms, games, and the internet , 2001, STOC '01.

[9]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[10]  Daphne Koller,et al.  A Continuation Method for Nash Equilibria in Structured Games , 2003, IJCAI.

[11]  Michael A. Goodrich,et al.  Learning To Cooperate in a Social Dilemma: A Satisficing Approach to Bargaining , 2003, ICML.

[12]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[13]  Y. Freund,et al.  Adaptive game playing using multiplicative weights , 1999 .

[14]  Andrew Chi-Chih Yao,et al.  Some complexity questions related to distributive computing(Preliminary Report) , 1979, STOC.

[15]  Vincent Conitzer,et al.  AWESOME: A general multiagent learning algorithm that converges in self-play and learns a best response against stationary opponents , 2003, Machine Learning.

[16]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[17]  Keith B. Hall,et al.  Correlated Q-Learning , 2003, ICML.

[18]  Manuela M. Veloso,et al.  Multiagent learning using a variable learning rate , 2002, Artif. Intell..