Algorithms for stochastic games — A survey

We consider finite state, finite action, stochastic games over an infinite time horizon. We survey algorithms for the computation of minimax optimal stationary strategies in the zerosum case, and of Nash equilibria in stationary strategies in the nonzerosum case. We also survey those theoretical results that pave the way towards future development of algorithms.ZusammenfassungIn dieser Arbeit werden unendlichstufige stochastische Spiele mit endlichen ZuStands- und Aktionenräumen untersucht. Es wird ein Überblick gegeben über Algorithmen zur Berechnung von optimalen stationären Minimax-Strategien in Nullsummen-Spielen und von stationären Nash-Gleichgewichtsstrategien in Nicht-Nullsummen-Spielen. Einige theoretische Ergebnisse werden vorgestellt, die für die weitere Entwicklung von Algorithmen nützlich sind.

[1]  Stef Tijs,et al.  Stochastic games with state independent transitions and separable rewards , 1984 .

[2]  Elon Kohlberg,et al.  The Asymptotic Theory of Stochastic Games , 1976, Math. Oper. Res..

[3]  Frank Thuijsman,et al.  Optimality and equilibria in stochastic games , 1992 .

[4]  Cyrus Derman,et al.  Finite State Markovian Decision Processes , 1970 .

[5]  O. J. Vrieze,et al.  Stochastic Games with Finite State and Action Spaces. , 1988 .

[6]  M. J. Sobel.  Noncooperative Stochastic Games , 1971 .

[7]  P. Schweitzer.  Perturbation theory and finite Markov chains , 1968 .

[8]  Jerzy A. Filar,et al.  A finite algorithm for the switching control stochastic game , 1983 .

[9]  O. J. Vrieze.  Linear programming and undiscounted stochastic games in which one player controls transitions , 1981 .

[10]  Jerzy A. Filar,et al.  Nonlinear programming and stationary strategies in stochastic games , 1986, Math. Program..

[11]  T. E. S. Raghavan,et al.  A Matrix Game Solution of the Single-Controller Stochastic Game , 1984, Math. Oper. Res..

[12]  S. Vajda.  Contributions to the Theory of Games. Volume III. Annals of Mathematics Studies Number 39. Edited by M. Dresher, A. W. Tucker and P. Wolfe. (Princeton University Press) , 1959 .

[13]  L. Shapley.  SOME TOPICS IN TWO-PERSON GAMES , 1963 .

[14]  O. J. Vrieze,et al.  On equilibria in repeated games with absorbing states , 1989 .

[15]  C. E. Lemke,et al.  Equilibrium Points of Bimatrix Games , 1964 .

[16]  S. R. Mohan,et al.  An algorithm for discounted Switching Control Stochastic Games , 1987 .

[17]  P. Gill,et al.  User's Guide for SOL/NPSOL: A Fortran Package for Nonlinear Programming. , 1983 .

[18]  Abraham Charnes,et al.  On some stocxastic tactical antisubmarine games , 1967 .

[19]  Elon Kohlberg,et al.  On Stochastic Games with Stationary Optimal Strategies , 1978, Math. Oper. Res..

[20]  C. E. Lemke,et al.  Bimatrix Equilibrium Points and Mathematical Programming , 1965 .

[21]  J. Neumann.  Zur Theorie der Gesellschaftsspiele , 1928 .

[22]  Jerzy Andrzej Filar,et al.  Algorithms for solving some undiscounted stochastic games , 1980 .

[23]  J. Robinson.  AN ITERATIVE METHOD OF SOLVING A GAME , 1951, Classics in Game Theory.

[24]  J. Filar.  Player aggregation in the traveling inspector model , 1985 .

[25]  M. Pollatschek,et al.  Algorithms for Stochastic Games with Geometrical Interpretation , 1969 .

[26]  Matthew J. Sobel,et al.  Myopic Solutions of Markov Decision Processes and Stochastic Games , 1981, Oper. Res..

[27]  Awi Federgruen.  Successive Approximation Methods in Undiscounted Stochastic Games , 1980, Oper. Res..

[28]  Jerzy A. Filar.  Quadratic programming and the single-controller stochastic game , 1986 .

[29]  O. J. Vrieze,et al.  On stochastic games with additive reward and transition structure , 1985 .

[30]  Dean Gillette,et al.  9. STOCHASTIC GAMES WITH ZERO STOP PROBABILITIES , 1958 .

[31]  A. M. Fink,et al.  Equilibrium in a stochastic $n$-person game , 1964 .

[32]  J. Filar.  Ordered field property for stochastic games when the player who controls transitions changes from state to state , 1981 .

[33]  Singiresu S. Rao,et al.  Algorithms for discounted stochastic games , 1973 .

[34]  Jerzy A. Filar,et al.  Bilinear programming and structured stochastic games , 1985 .

[35]  J. Wal.  Discounted Markov games; successive approximation and stopping times , 1977 .

[36]  Frank Thuijsman.  Non-zerosum stochastic games , 1987 .

[37]  J. Filar,et al.  On the Algorithm of Pollatschek and Avi-ltzhak , 1991 .

[38]  R. Karp,et al.  On Nonterminating Stochastic Games , 1966 .

[39]  Jerzy A. Filar,et al.  Nonlinear programming and stationary equilibria in stochastic games , 1991, Math. Program..

[40]  Melike Baykal-Gürsoy,et al.  A sample-path approach to stochastic games , 1989 .

[41]  Wayne L. Winston,et al.  A Stochastic Game Model of a Weapons Development Competition , 1978 .

[42]  T. Parthasarathy,et al.  An orderfield property for stochastic games when one player controls transition probabilities , 1981 .

[43]  Stef Tijs,et al.  Fictitious play applied to sequences of games and discounted stochastic games , 1982 .

[44]  A. Hordijk,et al.  A MODIFIED FORM OF THE ITERATIVE METHOD OF DYNAMIC PROGRAMMING , 1975 .

[45]  D. Blackwell,et al.  THE BIG MATCH , 1968, Classics in Game Theory.

[46]  Masayuki Takahashi.  Equilibrium points of stochastic non-cooperative $n$-person games , 1964 .

[47]  S. Lippman,et al.  Stochastic Games with Perfect Information and Time Average Payoff , 1969 .

[48]  J. Filar,et al.  On the computation of equilibria in discounted Stochastic games , 1986 .

[49]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.