Algorithms for uniform optimal strategies in two-player zero-sum stochastic games with perfect information

Abstract We deal with zero-sum two-player stochastic games with perfect information. We propose two algorithms to find the uniform optimal strategies and one method to compute the optimality range of discount factors. We prove the convergence in finite time for one algorithm. The uniform optimal strategies are also optimal for the long run average criterion and, in transient games, for the undiscounted criterion as well.

[1]  T. E. S. Raghavan,et al.  A policy-improvement type algorithm for solving zero-sum two-person stochastic games of perfect information , 2003, Math. Program..

[2]  T. Parthasarathy,et al.  An orderfield property for stochastic games when one player controls transition probabilities , 1981 .

[3]  Krishnendu Chatterjee,et al.  Stochastic limit-average games are in EXPTIME , 2008, Int. J. Game Theory.

[4]  Eitan Altman,et al.  An asymptotic simplex method for singularly perturbed linear programs , 2002, Oper. Res. Lett..

[5]  J. Filar.  Ordered field property for stochastic games when the player who controls transitions changes from state to state , 1981 .

[6]  Eitan Altman,et al.  Asymptotic linear programming and policy improvement for singularly perturbed Markov decision processes , 1999, Math. Methods Oper. Res..

[7]  K. Knopp.  Theory and Application of Infinite Series , 1990 .

[8]  T. Raghavan,et al.  Perfect information stochastic games and related classes , 1997 .

[9]  Robert G. Jeroslow,et al.  Asymptotic Linear Programming , 1973, Oper. Res..

[10]  Rommert Dekker,et al.  Sensitivity-analysis in discounted Markovian decision problems , 1985 .

[11]  E. Altman,et al.  Weighted Discounted Stochastic Games with Perfect Information , 2000 .

[12]  Nicolas Vieille,et al.  Computing uniformly optimal strategies in two-player stochastic games , 2009 .

[13]  David G. Luenberger,et al.  Linear and nonlinear programming , 1984 .

[14]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.

[15]  Dean Gillette,et al.  9. STOCHASTIC GAMES WITH ZERO STOP PROBABILITIES , 1958 .

[16]  Uriel G. Rothblum,et al.  Formulation of linear problems and solution by a universal machine , 1994, Math. Program..

[17]  Elon Kohlberg,et al.  The Asymptotic Theory of Stochastic Games , 1976, Math. Oper. Res..

[18]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[19]  Frank Thuijsman,et al.  Perfect information stochastic games and related classes , 1997, Int. J. Game Theory.

[20]  J. Filar,et al.  Competitive Markov Decision Processes , 1996 .