Stochastic games with additive transitions

Abstract We deal with n-player AT stochastic games, where AT stands for additive transitions. These are stochastic games in which the transition probability vector ps(as), for action combination a s = ( a s 1 , … , a s n ) in state s, can be decomposed into player-dependent components as: p s ( a s ) = ∑ i = 1 n λ s i · p s i ( a s i ) , where λ s i ∈ [ 0 , 1 ] for all players i, and ∑ i = 1 n λ s i = 1 , and where p s i ( a s i ) is a probability distribution on the finite set of states S. Here, λ s i reflects the influence of player i on the transitions in state s. As such the class of AT stochastic games covers several other well-known classes such as perfect information stochastic games, stochastic games with switching control, and so-called ARAT stochastic games. With respect to the average reward it is not clear whether e-equilibria always exist in general n-player stochastic games. For the class of n-player AT games we establish the existence of 0-equilibria, although the strategies involved may be history dependent. In addition we have the following results for the two-player case: (1) for zero-sum AT games, stationary 0-optimal strategies always exist; (2) for two-player general-sum AT absorbing games, there always exist stationary e-equilibria, for all e > 0. Several examples are provided to clarify the issues and to demonstrate the sharpness of the results.

[1]  S. Lippman,et al.  Stochastic Games with Perfect Information and Time Average Payoff , 1969 .

[2]  N. Vieille.  Two-player stochastic games I: A reduction , 2000 .

[3]  J. Filar.  Ordered field property for stochastic games when the player who controls transitions changes from state to state , 1981 .

[4]  O. J. Vrieze,et al.  Simplifying Optimal Strategies in Stochastic Games , 1998 .

[5]  János Flesch,et al.  Recursive Repeated Games with Absorbing States , 1996, Math. Oper. Res..

[6]  Arie Hordijk,et al.  Semi-markov strategies in stochastic games , 1983 .

[7]  Chris Cannings,et al.  Stochastic Games and Related Topics , 1991 .

[8]  T. E. S. Raghavan,et al.  Repeated ARAT games , 1996 .

[9]  O. J. Vrieze,et al.  On equilibria in repeated games with absorbing states , 1989 .

[10]  A. M. Fink,et al.  Equilibrium in a stochastic $n$-person game , 1964 .

[11]  Dean Gillette,et al.  9. STOCHASTIC GAMES WITH ZERO STOP PROBABILITIES , 1958 .

[12]  Masayuki Takahashi.  Equilibrium points of stochastic non-cooperative $n$-person games , 1964 .

[13]  S. Sorin.  Asymptotic properties of a non-zero sum stochastic game , 1986 .

[14]  J. Doob.  Stochastic processes , 1953 .

[15]  Frank Thuijsman,et al.  Perfect information stochastic games and related classes , 1997, Int. J. Game Theory.

[16]  János Flesch,et al.  Cyclic Markov equilibria in stochastic games , 1997, Int. J. Game Theory.

[17]  P. Dutta.  A Folk Theorem for Stochastic Games , 1995 .

[18]  O. J. Vrieze,et al.  On stochastic games with additive reward and transition structure , 1985 .

[19]  Nicolas Vieille,et al.  Two-player stochastic games II: The case of recursive games , 2000 .

[20]  O. J. Vrieze,et al.  Easy initial states in stochastic games , 1991 .

[21]  Stef Tijs,et al.  On the Existence of Easy Initial States for Undiscounted Stochastic Games , 1986, Math. Oper. Res..

[22]  Julia Donaldson,et al.  The big match , 2008 .

[23]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.