The Gauss-Seidel numerical procedure for Markov stochastic games

Consider the problem of value iteration for solving Markov stochastic games. One simply iterates backward, via a Jacobi-like procedure. The convergence of the Gauss-Seidel form of this procedure is shown for both the discounted and ergodic cost problems, under appropriate conditions, with extensions to problems where one stops when a boundary is hit or if any one of the players chooses to stop, with associated costs. Generally, the Gauss-Seidel procedure accelerates convergence.

[1]  Harold J. Kushner.  Numerical Approximations for Stochastic Differential Games: The Ergodic Case , 2004, SIAM J. Control. Optim..

[2]  J. Filar,et al.  On the computation of equilibria in discounted Stochastic games , 1986 .

[3]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.

[4]  D. White,et al.  Dynamic programming, Markov chains, and the method of successive approximations , 1963 .

[5]  K. Åström.  Introduction to Stochastic Control Theory , 1970 .

[6]  H. Kushner,et al.  Numerical methods for the solution of the degenerate nonlinear elliptic equations arising in optimal stochastic control theory , 1968 .

[7]  Harold J. Kushner,et al.  On stochastic differential games: Sufficient conditions that a given strategy be a saddle point, and numerical procedures for the solution of the game☆ , 1969 .

[8]  Boleslaw Tolwinski.  Solving dynamic games via Markov game approximations , 1991 .

[9]  H. Kushner.  Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[10]  R. Karp,et al.  On Nonterminating Stochastic Games , 1966 .

[11]  Dimitri P. Bertsekas,et al.  Dynamic Programming: Deterministic and Stochastic Models , 1987 .

[12]  H. Kushner,et al.  Finite state stochastic games: Existence theorems and computational procedures , 1969 .

[13]  Pierre L'Ecuyer,et al.  Approximate solutions to continuous stochastic games , 1990 .

[14]  HAROLD J. KUSHNER,et al.  Numerical Approximations for Stochastic Differential Games , 2002, SIAM J. Control. Optim..

[15]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[16]  J. Filar,et al.  Competitive Markov Decision Processes , 1996 .