The Complexity of Ergodic Mean-payoff Games

We study two-player (zero-sum) concurrent mean-payoff games played on a finite-state graph. We focus on the important sub-class of ergodic games where all states are visited infinitely often with probability 1. The algorithmic study of ergodic games was initiated in a seminal work of Hoffman and Karp in 1966, but all basic complexity questions have remained unresolved. Our main results for ergodic games are as follows: We establish (1) an optimal exponential bound on the patience of stationary strategies (where patience of a distribution is the inverse of the smallest positive probability and represents a complexity measure of a stationary strategy); (2) the approximation problem lies in FNP; (3) the approximation problem is at least as hard as the decision problem for simple stochastic games (for which NP ∩ coNP is the long-standing best known bound). We present a variant of the strategy-iteration algorithm by Hoffman and Karp; show that both our algorithm and the classical value-iteration algorithm can approximate the value in exponential time; and identify a subclass where the value-iteration algorithm is a FPTAS. We also show that the exact value can be expressed in the existential theory of the reals, and establish square-root sum hardness for a related class of games.

[1]  Wolfgang Thomas,et al.  Languages, Automata, and Logic , 1997, Handbook of Formal Languages.

[2]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, STOC '84.

[3]  Khaled M. Elbassioni,et al.  A potential reduction algorithm for two-person zero-sum limiting average payoff stochastic games , 2012 .

[4]  Moshe Y. Vardi.  Automatic verification of probabilistic concurrent finite state programs , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[5]  Kristoffer Arnsfelt Hansen,et al.  Exact algorithms for solving stochastic games: extended abstract , 2011, STOC '11.

[6]  Elon Kohlberg,et al.  The Asymptotic Theory of Stochastic Games , 1976, Math. Oper. Res..

[7]  Kristoffer Arnsfelt Hansen,et al.  The Complexity of Solving Reachability Games Using Value and Strategy Iteration , 2011, CSR.

[8]  R. Karp,et al.  On Nonterminating Stochastic Games , 1966 .

[9]  Kristoffer Arnsfelt Hansen,et al.  Exact Algorithms for Solving Stochastic Games , 2012, ArXiv.

[10]  Julia Donaldson,et al.  The big match , 2008 .

[11]  Rupak Majumdar,et al.  Quantitative solution of omega-regular games380872 , 2001, STOC '01.

[12]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.

[13]  Krishnendu Chatterjee,et al.  Stochastic limit-average games are in EXPTIME , 2008, Int. J. Game Theory.

[14]  U. Rieder,et al.  Markov Decision Processes , 2010 .

[15]  Peter Bro Miltersen,et al.  On the Complexity of Numerical Analysis , 2009, SIAM J. Comput..

[16]  Dean Gillette,et al.  9. STOCHASTIC GAMES WITH ZERO STOP PROBABILITIES , 1958 .

[17]  Rupak Majumdar,et al.  Quantitative solution of omega-regular games , 2004, J. Comput. Syst. Sci..

[18]  John Fearnley,et al.  Exponential Lower Bounds for Policy Iteration , 2010, ICALP.

[19]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[20]  Singiresu S. Rao,et al.  Algorithms for discounted stochastic games , 1973 .

[21]  D. Aldous.  Random walks on finite groups and rapidly mixing markov chains , 1983 .

[22]  Peter Bro Miltersen,et al.  On the complexity of numerical analysis , 2006, 21st Annual IEEE Conference on Computational Complexity (CCC'06).

[23]  Vladimir Gurvich,et al.  A Potential Reduction Algorithm for Ergodic Two-Person Zero-Sum Limiting Average Payoff Stochastic Games , 2014, COCOA.

[24]  Anne Condon,et al.  The Complexity of Stochastic Games , 1992, Inf. Comput..

[25]  Oliver Friedmann,et al.  An Exponential Lower Bound for the Latest Deterministic Strategy Iteration Algorithms , 2011, Log. Methods Comput. Sci..

[26]  Kurt Mehlhorn,et al.  Optimal search for rationals , 2003, Inf. Process. Lett..

[27]  Hendrik W. Lenstra,et al.  Integer Programming with a Fixed Number of Variables , 1983, Math. Oper. Res..

[28]  J. Filar,et al.  Competitive Markov Decision Processes , 1996 .

[29]  Ludwig Staiger,et al.  Ω-languages , 1997 .

[30]  Peter Bro Miltersen,et al.  Solving simple stochastic games with few coin toss positions , 2012, ESA.

[31]  Marie-Françoise Roy,et al.  Existential Theory of the Reals , 2003 .

[32]  H. Everett.  2. RECURSIVE GAMES , 1958 .

[33]  Uri Zwick,et al.  The Complexity of Mean Payoff Games on Graphs , 1996, Theor. Comput. Sci..

[34]  Kousha Etessami,et al.  On the Complexity of Nash Equilibria and Other Fixed Points , 2010, SIAM J. Comput..

[35]  Kousha Etessami,et al.  Recursive Concurrent Stochastic Games , 2008, Log. Methods Comput. Sci..

[36]  Kristoffer Arnsfelt Hansen,et al.  Winning Concurrent Reachability Games Requires Doubly-Exponential Patience , 2009, 2009 24th Annual IEEE Symposium on Logic In Computer Science.

[37]  John F. Canny,et al.  Some algebraic and geometric computations in PSPACE , 1988, STOC '88.

[38]  Kristoffer Arnsfelt Hansen,et al.  The Complexity of Solving Reachability Games Using Value and Strategy Iteration , 2013, Theory of Computing Systems.