The Complexity of Solving Reachability Games Using Value and Strategy Iteration

Two standard algorithms for approximately solving two-player zero-sum concurrent reachability games are value iteration and strategy iteration. We prove upper and lower bounds of $2^{m^{\varTheta(N)}}$ on the worst case number of iterations needed by both of these algorithms for providing non-trivial approximations to the value of a game with N non-terminal positions and m actions for each player in each position. In particular, both algorithms have doubly-exponential complexity. Even when the game given as input has only one non-terminal position, we prove an exponential lower bound on the worst case number of iterations needed to provide non-trivial approximations.

[1]  Krishnendu Chatterjee,et al.  Strategy Improvement for Concurrent Safety Games , 2008, ArXiv.

[2]  Anne Condon,et al.  On Algorithms for Simple Stochastic Games , 1990, Advances In Computational Complexity Theory.

[3]  Rupak Majumdar,et al.  Quantitative solution of omega-regular games , 2004, J. Comput. Syst. Sci..

[4]  Kristoffer Arnsfelt Hansen,et al.  The Complexity of Solving Reachability Games Using Value and Strategy Iteration , 2011, CSR.

[5]  Thomas A. Henzinger,et al.  Concurrent reachability games , 2007, Theor. Comput. Sci..

[6]  Singiresu S. Rao,et al.  Algorithms for discounted stochastic games , 1973 .

[7]  T. Parthasarathy.  Discounted and positive stochastic games , 1971 .

[8]  Krishnendu Chatterjee,et al.  Termination criteria for solving concurrent safety and reachability games , 2009, SODA.

[9]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.

[10]  Krishnendu Chatterjee,et al.  On Nash Equilibria in Stochastic Games , 2004, CSL.

[11]  Kristoffer Arnsfelt Hansen,et al.  Exact algorithms for solving stochastic games: extended abstract , 2011, STOC.

[12]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[13]  Krishnendu Chatterjee,et al.  Strategy Improvement for Concurrent Reachability Games , 2006, Third International Conference on the Quantitative Evaluation of Systems - (QEST'06).

[14]  C. J. Himmelberg,et al.  Existence of p-equilibrium and optimal stationary strategies in stochastic games , 1976 .

[15]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[16]  Kousha Etessami,et al.  Recursive Concurrent Stochastic Games , 2008, Log. Methods Comput. Sci..

[17]  Rong Ge,et al.  New Results on Simple Stochastic Games , 2009, ISAAC.

[18]  Oliver Friedmann,et al.  An Exponential Lower Bound for the Parity Game Strategy Improvement Algorithm as We Know it , 2009, 2009 24th Annual IEEE Symposium on Logic In Computer Science.

[19]  R. Karp,et al.  On Nonterminating Stochastic Games , 1966 .

[20]  Kristoffer Arnsfelt Hansen,et al.  Winning Concurrent Reachability Games Requires Doubly-Exponential Patience , 2009, 2009 24th Annual IEEE Symposium on Logic In Computer Science.