How to solve large scale deterministic games with mean payoff by policy iteration

Min-max functions are dynamic programming operators of zero-sum deterministic games with finite state and action spaces. The problem of computing the linear growth rate of the orbits (cycle-time) of a min-max function, which is equivalent to computing the value of a deterministic game with mean payoff, arises in the performance analysis of discrete event systems. We present here an improved version of the policy iteration algorithm given by Gaubert and Gunawardena in 1998 to compute the cycle-time of a min-max functions. The improvement consists of a fast evaluation of the spectral projector which is adapted to the case of large sparse graphs. We present detailed numerical experiments, both on randomly generated instances, and on concrete examples, indicating that the algorithm is experimentally fast.

[1]  A. Ehrenfeucht,et al.  Positional strategies for mean payoff games , 1979 .

[2]  J. Quadrat,et al.  Analyse du comportement periodique de systemes de production par la theorie des dioides , 1983 .

[3]  Takuji Nishimura,et al.  Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator , 1998, TOMC.

[4]  S. Gaubert,et al.  THE DUALITY THEOREM FOR MIN-MAX FUNCTIONS , 1997 .

[5]  Uri Zwick,et al.  The Complexity of Mean Payoff Games on Graphs , 1996, Theor. Comput. Sci..

[6]  Henrik Björklund,et al.  A combinatorial strongly subexponential strategy improvement algorithm for mean payoff games , 2007, Discret. Appl. Math..

[7]  Eric Goubault,et al.  A Policy Iteration Algorithm for Computing Fixed Points in Static Analysis of Programs , 2005, CAV.

[8]  M. Paterson,et al.  A deterministic subexponential algorithm for solving parity games , 2006, SODA 2006.

[9]  R. Karp,et al.  On Nonterminating Stochastic Games , 1966 .

[10]  Da-Zhong Zheng,et al.  A Cycle Time Computing Algorithm and its Application in the Structural Analysis of Min-max Systems , 2004, Discret. Event Dyn. Syst..

[11]  Geert Jan Olsder,et al.  Eigenvalues of dynamic max-min systems , 1991, Discret. Event Dyn. Syst..

[12]  Jeremy Gunawardena,et al.  Min-max functions , 1994, Discret. Event Dyn. Syst..

[13]  Anne Condon,et al.  The Complexity of Stochastic Games , 1992, Inf. Comput..

[14]  A. Karzanov,et al.  Cyclic games and an algorithm to find minimax cycle means in directed graphs , 1990 .

[15]  Marcin Jurdzinski,et al.  A Discrete Strategy Improvement Algorithm for Solving Parity Games , 2000, CAV.

[16]  C. Leake.  Synchronization and Linearity: An Algebra for Discrete Event Systems , 1994 .

[17]  J. Quadrat,et al.  Numerical Computation of Spectral Elements in Max-Plus Algebra☆ , 1998 .

[18]  Elon Kohlberg,et al.  Invariant Half-Lines of Nonexpansive Piecewise-Linear Transformations , 1980, Math. Oper. Res..

[19]  R. Bellman,et al.  Dynamic Programming and Markov Processes , 1960 .

[20]  Richard M. Karp,et al.  A characterization of the minimum cycle mean in a digraph , 1978, Discret. Math..

[21]  Ricardo Katz.  Max-Plus $(A,B)$-Invariant Spaces and Control of Timed Discrete-Event Systems , 2007, IEEE Transactions on Automatic Control.

[22]  Jean Cochet-Terrasson.  A constructive xed point theorem for min-max functions , 1999 .

[23]  Ganesh Gopalakrishnan,et al.  Performance analysis and optimization of asynchronous circuits , 1994, Proceedings 1994 IEEE International Conference on Computer Design: VLSI in Computers and Processors.

[24]  Elon Kohlberg,et al.  The Asymptotic Solution of a Recursion Equation Occurring in Stochastic Games , 1976, Math. Oper. Res..