Simplifying optimal strategies in limsup and liminf stochastic games

Abstract We consider two-player zero-sum stochastic games with the limsup and with the liminf payoffs. For the limsup payoff, we prove that the existence of an optimal strategy implies the existence of a stationary optimal strategy. Our construction does not require the knowledge of an optimal strategy, only its existence. The main technique of the proof is to analyze the game with specific restricted action spaces. For the liminf payoff, we prove that the existence of a subgame-optimal strategy (i.e. a strategy that is optimal in every subgame) implies the existence of a subgame-optimal strategy under which the prescribed mixed actions only depend on the current state and on the state and the actions chosen at the previous period. In particular, such a strategy requires only finite memory. The proof relies on techniques that originate in gambling theory.

[1]  Zohar Manna,et al.  Formal verification of probabilistic systems , 1997 .

[2]  W. Sudderth,et al.  Discrete Gambling and Stochastic Games , 1996 .

[3]  Hugo Gimbert,et al.  Pure Stationary Optimal Strategies in Markov Decision Processes , 2007, STACS.

[4]  Sylvain Sorin,et al.  On the values of repeated games with signals , 2014, ArXiv.

[5]  P. Secchi.  On the existence of good stationary strategies for nonleavable stochastic games , 1998 .

[6]  M. Orkin.  On Stationary Policies--The General Case , 1974 .

[7]  Krishnendu Chatterjee,et al.  A Survey of Stochastic Games with Limsup and Liminf Objectives , 2009, ICALP.

[8]  János Flesch,et al.  Markov Strategies are Better than stationary Strategies , 1999, IGTR.

[9]  David Lindley,et al.  How to Gamble if You Must. (Inequalities for Stochastic Processes) By Lester E. Dubins and Leonard J. Savage. Pp. xiv, 249. 102s. 1965. (McGraw-Hill Book Co.) , 1966, The Mathematical Gazette.

[10]  A. Maitra,et al.  STOCHASTIC GAMES WITH BOREL PAYOFFS , 2003 .

[11]  Véronique Bruyère,et al.  Computer Aided Synthesis: A Game-Theoretic Approach , 2017, DLT.

[12]  O. J. Vrieze,et al.  Simplifying Optimal Strategies in Stochastic Games , 1998 .

[13]  A. Maitra,et al.  Borel Stochastic Games with Lim Sup Payoff , 1993 .

[14]  Dean Gillette,et al.  9. STOCHASTIC GAMES WITH ZERO STOP PROBABILITIES , 1958 .

[15]  D. Blackwell,et al.  THE BIG MATCH , 1968, Classics in Game Theory.

[16]  Krishnendu Chatterjee,et al.  A survey of stochastic ω-regular games , 2012, J. Comput. Syst. Sci..

[17]  William D. Sudderth,et al.  An operator solution of stochastic games , 1992 .

[18]  Donald A. Martin,et al.  The determinacy of Blackwell games , 1998, Journal of Symbolic Logic.

[19]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.

[20]  William D. Sudderth.  Gambling Problems with a Limit Inferior Payoff , 1983, Math. Oper. Res..

[21]  David Blackwell,et al.  On Stationary Policies , 1970 .