The complexity of all-switches strategy improvement

Strategy improvement is a widely-used and well-studied class of algorithms for solving graph-based infinite games. These algorithms are parametrized by a switching rule, and one of the most natural rules is "all switches" which switches as many edges as possible in each iteration. Continuing a recent line of work, we study all-switches strategy improvement from the perspective of computational complexity. We consider two natural decision problems, both of which have as input a game G, a starting strategy s, and an edge e. The problems are: 1. The edge switch problem, namely, is the edge e ever switched by all-switches strategy improvement when it is started from s on game G? 2. The optimal strategy problem, namely, is the edge e used in the final strategy that is found by strategy improvement when it is started from s on game G? We show PSPACE-completeness of the edge switch problem and optimal strategy problem for the following settings: Parity games with the discrete strategy improvement algorithm of Voge and Jurdzinski; mean-payoff games with the gain-bias algorithm [11, 33]; and discounted-payoff games and simple stochastic games with their standard strategy improvement algorithms. We also show PSPACE-completeness of an analogous problem to edge switch for the bottom-antipodal algorithm for Acyclic Unique Sink Orientations on Cubes.

[1]  Walter Ludwig,et al.  A Subexponential Randomized Algorithm for the Simple Stochastic Game Problem , 1995, Inf. Comput..

[2]  J. Filar,et al.  Competitive Markov Decision Processes , 1996 .

[3]  Anne Condon,et al.  On Algorithms for Simple Stochastic Games , 1990, Advances In Computational Complexity Theory.

[4]  Tibor Szabó,et al.  Unique sink orientations of cubes , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[5]  Jirí Matousek,et al.  Random edge can be exponential on abstract cubes , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[6]  A. Puri.  Theory of hybrid systems and discrete event systems , 1996 .

[7]  John Fearnley,et al.  Exponential Lower Bounds for Policy Iteration , 2010, ICALP.

[8]  Oliver Friedmann,et al.  An Exponential Lower Bound for the Latest Deterministic Strategy Iteration Algorithms , 2011, Log. Methods Comput. Sci..

[9]  Uri Zwick,et al.  Subexponential lower bounds for randomized pivoting rules for the simplex algorithm , 2011, STOC '11.

[10]  Tibor Szabó,et al.  Jumping Doesn't Help in Abstract Cubes , 2005, IPCO.

[11]  Bernd Gärtner,et al.  Unique Sink Orientations of Grids , 2005, Algorithmica.

[12]  Gil Kalai,et al.  A subexponential randomized simplex algorithm (extended abstract) , 1992, STOC '92.

[13]  Micha Sharir,et al.  A subexponential bound for linear programming , 1992, SCG '92.

[14]  A. Prasad Sistla,et al.  On Model-Checking for Fragments of µ-Calculus , 1993, CAV.

[15]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[16]  John Fearnley,et al.  The Complexity of the Simplex Method , 2015, STOC.

[17]  Rahul Savani,et al.  A Simple P-Matrix Linear Complementarity Problem for Discounted Games , 2008, CiE.

[18]  Uri Zwick,et al.  The Complexity of Mean Payoff Games on Graphs , 1996, Theor. Comput. Sci..

[19]  John Fearnley,et al.  Non-oblivious Strategy Improvement , 2010, LPAR.

[20]  Marcin Jurdziński,et al.  Deciding the Winner in Parity Games is in UP \cap co-Up , 1998, Inf. Process. Lett..

[21]  Uri Zwick,et al.  Improved upper bounds for Random-Edge and Random-Jump on abstract cubes , 2014, SODA.

[22]  R. Karp,et al.  On Nonterminating Stochastic Games , 1966 .

[23]  Mihalis Yannakakis,et al.  How easy is local search? , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[24]  Anne Condon,et al.  The Complexity of Stochastic Games , 1992, Inf. Comput..

[25]  Paul W. Goldberg,et al.  The Complexity of the Homotopy Method, Equilibrium Selection, and Lemke-Howson Solutions , 2010, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[26]  Oliver Friedmann,et al.  A Subexponential Lower Bound for Zadeh's Pivoting Rule for Solving Linear Programs and Games , 2011, IPCO.

[27]  Bernd Gärtner,et al.  Linear programming and unique sink orientations , 2006, SODA '06.

[28]  Sven Schewe,et al.  An Optimal Strategy Improvement Algorithm for Solving Parity and Payoff Games , 2008, CSL.

[29]  Martin Skutella,et al.  The Simplex Algorithm is NP-mighty , 2015, SODA.

[30]  Marcin Jurdzinski,et al.  A Discrete Strategy Improvement Algorithm for Solving Parity Games , 2000, CAV.

[31]  Colin Stirling,et al.  Lokal Model Checking Games , 1995, CONCUR.

[32]  John Fearnley,et al.  Linear Complementarity Algorithms for Infinite Games , 2009, SOFSEM.

[33]  M. Paterson,et al.  A deterministic subexponential algorithm for solving parity games , 2006, SODA 2006.

[34]  Christos H. Papadimitriou,et al.  On the Complexity of the Parity Argument and Other Inefficient Proofs of Existence , 1994, J. Comput. Syst. Sci..

[35]  Christos H. Papadimitriou,et al.  On Simplex Pivoting Rules and Complexity Theory , 2014, IPCO.

[36]  Uri Zwick,et al.  A subexponential lower bound for the random facet algorithm for parity games , 2011, SODA '11.

[37]  S. Smale.  Mathematical problems for the next century , 1998 .

[38]  E. Allen Emerson,et al.  Tree automata, mu-calculus and determinacy , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.