On stationary equilibria of a single-controller stochastic game

We consider a two-person, general-sum, rational-data, undiscounted stochastic game in which one player (player II) controls the transition probabilities. We show that the set of stationary equilibrium points is the union of a finite number of sets such that, every element of each of these sets can be constructed from a finite number of extreme equilibrium strategies for player I and from a finite number of pseudo-extreme equilibrium strategies for player II. These extreme and pseudo-extreme strategies can themselves be constructed by finite (but inefficient) algorithms. Analogous results can also be established in the more straightforward case of discounted single-controller games.

[1]  E. Denardo,et al.  Multichain Markov Renewal Programs , 1968 .

[2]  J F Mertens,et al.  Stochastic games have a value. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[3]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.

[4]  Elon Kohlberg,et al.  On Stochastic Games with Stationary Optimal Strategies , 1978, Math. Oper. Res..

[5]  H W Kuhn,et al.  AN ALGORITHM FOR EQUILIBRIUM POINTS IN BIMATRIX GAMES. , 1961, Proceedings of the National Academy of Sciences of the United States of America.

[6]  O. J. Vrieze Linear programming and undiscounted stochastic games in which one player controls transitions , 1981 .

[7]  T. E. S. Raghavan,et al.  A Matrix Game Solution of the Single-Controller Stochastic Game , 1984, Math. Oper. Res..

[8]  A. Hordijk,et al.  Linear Programming and Markov Decision Chains , 1979 .

[9]  E. Kohlberg Repeated Games with Absorbing States , 1974 .

[10]  D. Blackwell,et al.  THE BIG MATCH , 1968, Classics in Game Theory.

[11]  Jerzy Andrzej Filar,et al.  Algorithms for solving some undiscounted stochastic games , 1980 .

[12]  N. Vorob’ev Equilibrium Points in Bimatrix Games , 1958 .

[13]  T. Parthasarathy,et al.  An orderfield property for stochastic games when one player controls transition probabilities , 1981 .

[14]  A. J. Goldman 3 . Resolution and Separation Theorems for Polyhedral Convex Sets , 1957 .

[15]  M. Balinski An algorithm for finding all vertices of convex polyhedral sets , 1959 .

[16]  R. Karp,et al.  On Nonterminating Stochastic Games , 1966 .

[17]  Dean Gillette,et al.  9. STOCHASTIC GAMES WITH ZERO STOP PROBABILITIES , 1958 .