From Markov Chains to Stochastic Games

Markov chains1 and Markov decision processes (MDPs) are special cases of stochastic games. Markov chains describe the dynamics of the states of a stochastic game where each player has a single action in each state. Similarly, the dynamics of the states of a stochastic game form a Markov chain whenever the players’ strategies are stationary. Markov decision processes are stochastic games with a single player. In addition, the decision problem faced by a player in a stochastic game when all other players choose a fixed profile of stationary strategies is equivalent to an MDP.

[1]  G. H. Hardy,et al.  Notes on the Theory of Series (XVI): Two Tauberian Theorems , 1931 .

[2]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.

[3]  John G. Kemeny,et al.  Finite Markov Chains. , 1960 .

[4]  D. Blackwell.  Discrete Dynamic Programming , 1962 .

[5]  R. Bellman.  Dynamic programming. , 1957, Science.

[6]  Elon Kohlberg,et al.  The Asymptotic Solution of a Recursion Equation Occurring in Stochastic Games , 1976, Math. Oper. Res..

[7]  Elon Kohlberg,et al.  The Asymptotic Theory of Stochastic Games , 1976, Math. Oper. Res..

[8]  John G. Kemeny,et al.  Finite Markov chains , 1960 .

[9]  D. J. White.  Markov Decision Processes , 2006 .

[10]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[11]  J. Filar,et al.  Competitive Markov Decision Processes , 1996 .

[12]  Nicolas Vielle.  Small perturbations and stochastic games , 2000 .

[13]  N. Vieille.  Small perturbations and stochastic games , 2000 .

[14]  Nicolas Vieille,et al.  Correlated Equilibrium in Stochastic Games , 2002, Games Econ. Behav..

[15]  Sylvain Sorin,et al.  DISCOUNTED STOCHASTIC GAMES: THE FINITE CASE , 2003 .

[16]  Eilon Solan.  Perturbations of Markov Chains with Applications to Stochastic Games , 2003 .

[17]  O. J. Vrieze.  Stochastic Games and Stationary Strategies , 2003 .

[18]  Abraham Neyman,et al.  Stochastic games and nonexpansive maps , 2003 .

[19]  Abraham Neyman.  Stochastic games: Existence of the MinMax , 2003 .

[20]  T. Raghavan,et al.  Finite-Step Algorithms for Single-Controller and Perfect Information Stochastic Games , 2003 .

[21]  Abraham Neyman,et al.  Real Algebraic Tools in Stochastic Games , 2003 .

[22]  O. J. Vrieze.  Stochastic Games, Practical Motivation and the Orderfield Property for Special Classes , 2003 .

[23]  Sylvain Sorin,et al.  Stochastic Games and Applications , 2003 .