Computing uniformly optimal strategies in two-player stochastic games

We provide a computable algorithm to calculate uniform ε-optimal strategies in two-player zero-sum stochastic games. Our approach can be used to construct algorithms that calculate uniform ε-equilibria and uniform correlated ε-equilibria in various classes of multi-player non-zero-sum stochastic games.

[1]  Stef Tijs,et al.  Fictitious play applied to sequences of games and discounted stochastic games , 1982 .

[2]  P. Jean-Jacques Herings,et al.  Stationary Equilibria in Stochastic Games: Structure, Selection and Computation , 2000 .

[3]  Marie-Françoise Roy,et al.  Real algebraic geometry , 1992 .

[4]  Uriel G. Rothblum,et al.  A Theory on Extending Algorithms for Parametric Problems , 1989, Math. Oper. Res..

[5]  A. M. Fink,et al.  Equilibrium in a stochastic $n$-person game , 1964 .

[6]  Perturbed Markov chains , 2003, Journal of Applied Probability.

[7]  Abraham Neyman.  Stochastic games: Existence of the MinMax , 2003 .

[8]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.

[9]  Julia Donaldson,et al.  The big match , 2008 .

[10]  Elon Kohlberg,et al.  The Asymptotic Theory of Stochastic Games , 1976, Math. Oper. Res..

[11]  János Flesch,et al.  Stochastic Games on a Product State Space , 2008, Math. Oper. Res..

[12]  J. Filar,et al.  Competitive Markov Decision Processes , 1996 .

[13]  T. Parthasarathy,et al.  An orderfield property for stochastic games when one player controls transition probabilities , 1981 .

[14]  O. J. Vrieze,et al.  On equilibria in repeated games with absorbing states , 1989 .

[15]  János Flesch,et al.  Stochastic games with additive transitions , 2007, Eur. J. Oper. Res..

[16]  Eitan Altman,et al.  Asymptotic linear programming and policy improvement for singularly perturbed Markov decision processes , 1999, Math. Methods Oper. Res..

[17]  Rakesh V. Vohra,et al.  Correlated equilibrium payoffs and public signalling in absorbing games , 2002, Int. J. Game Theory.

[18]  Nicolas Vieille,et al.  Quitting Games , 2001, Math. Oper. Res..

[19]  Ronald Peeters,et al.  StochasticGameSolver: Computing Stationary Equilibria for Stochastic Games , 2004 .

[20]  Nicolas Vieille,et al.  Correlated Equilibrium in Stochastic Games , 2002, Games Econ. Behav..

[21]  D. S. Arnon,et al.  Algorithms in real algebraic geometry , 1988 .

[22]  Saugata Basu,et al.  New results on quantifier elimination over real closed fields and applications to constraint databases , 1999, JACM.

[23]  Nicolas Vieille,et al.  Two-player stochastic games II: The case of recursive games , 2000 .

[24]  J. Risler,et al.  Real algebraic and semi-algebraic sets , 1990 .

[25]  Eilon Solan.  Continuity of the Value of Competitive Markov Decision Processes , 2003 .

[26]  A. Tarski.  A Decision Method for Elementary Algebra and Geometry , 2023 .

[27]  Eilon Solan.  Three-Player Absorbing Games , 1999, Math. Oper. Res..

[28]  Lance Fortnow,et al.  Beating a Finite Automaton in the Big Match , 1998, TARK.

[29]  N. Vieille.  Two-player stochastic games I: A reduction , 2000 .

[30]  Sylvain Sorin,et al.  Stochastic Games and Applications , 2003 .

[31]  Krishnendu Chatterjee,et al.  Stochastic limit-average games are in EXPTIME , 2008, Int. J. Game Theory.