Stationary Equilibria in Stochastic Games: Structure, Selection and Computation

This paper is the first to introduce an algorithm to compute stationary equilibria in stochastic games, and shows convergence of the algorithm for almost all such games. Moreover, since in general the number of stationary equilibria is overwhelming, we pay attention to the issue of equilibrium selection. We do this by extending the linear tracing procedure to the class of stochastic games, called the stochastic tracing procedure. From a computational point of view, the class of stochastic games possesses substantial difficulties compared to normal form games. Apart from technical difficulties, there are also conceptual difficulties,, for instance the question how to extend the linear tracing procedure to the environment of stochastic games. We prove that there is a generic subclass of the class of stochastic games for which the stochastic tracing procedure is a compact one-dimensional piecewise differentiable manifold with boundary. Furthermore, we prove that the stochastic tracing procedure generates a unique path leading from any exogenously specified prior belief, to a stationary equilibrium. A well-chosen transformation of variables is used to formulate an everywhere differentiable homotopy function, whose zeros describe the (unique) path generated by the stochastic tracing procedure. Because of differentiability we are able to follow this path using standard path-following techniques. This yields a globally convergent algorithm that is easily and robustly implemented on a computer using existing software routines. As a by-product of our results, we extend a recent result on the generic finiteness of stationary equilibria in stochastic games to oddness of equilibria.

[1]  Layne T. Watson,et al.  Algorithm 555: Chow-Yorke Algorithm for Fixed Points or Zeros of C2 Maps [C5] , 1980, TOMS.

[2]  Jerzy A. Filar,et al.  Nonlinear programming and stationary equilibria in stochastic games , 1991, Math. Program..

[3]  R. McKelvey,et al.  Quantal Response Equilibria for Extensive Form Games , 1998 .

[4]  R. Kellogg,et al.  Pathways to solutions, fixed points, and equilibria , 1983 .

[5]  Robert Wilson,et al.  Computing Equilibria of N-Person Games , 1971 .

[6]  A.J.J. Talman,et al.  Tracing equilibria in extensive games by complementary pivoting , 1996 .

[7]  Ariel Pakes,et al.  The Dynamics of Productivity in the Telecommunications Equipment Industry , 1996 .

[8]  Ronald Peeters,et al.  A differentiable homotopy to compute Nash equilibria of n-person games , 2000 .

[9]  A. McLennan.  The Expected Number of Nash Equilibria of a Normal Form Game , 2005 .

[10]  J. Rosenmüller.  On a Generalization of the Lemke–Howson Algorithm to Noncooperative N-Person Games , 1971 .

[11]  John C. Harsanyi,et al.  Общая теория выбора равновесия в играх / A General Theory of Equilibrium Selection in Games , 1989 .

[12]  P. Jean-Jacques Herings,et al.  Computation of the Nash Equilibrium Selected by the Tracing Procedure in N-Person Games , 2002, Games Econ. Behav..

[13]  P. Herings,et al.  A globally and universally stable price adjustment process , 1997 .

[14]  Rs Stainton,et al.  Pathways to solutions , 1984 .

[15]  A. Pakes,et al.  The Dynamics of Productivity in the Telecommunications Equipment Industry , 1992 .

[16]  Bernhard von Stengel,et al.  Computing Normal Form Perfect Equilibria for Extensive Two-Person Games , 2002 .

[17]  Roger Lagunoff,et al.  Genericity and Markovian Behavior in Stochastic Games , 2000 .

[18]  Eric Maskin,et al.  Markov Perfect Equilibrium: I. Observable Actions , 2001, J. Econ. Theory.

[19]  D. Bergemann,et al.  Learning and Strategic Pricing , 1996 .

[20]  R. McKelvey,et al.  Computation of equilibria in finite games , 1996 .

[21]  J. Harsanyi.  Oddness of the number of equilibrium points: A new proof , 1973 .

[22]  Herbert E. Scarf,et al.  The Approximation of Fixed Points of a Continuous Mapping , 1967 .

[23]  A. Mas-Colell.  The Theory Of General Economic Equilibrium , 1985 .

[24]  Dolf Talman,et al.  An Algorithmic Approach toward the Tracing Procedure for Bi-matrix Games , 1999 .

[25]  P. Herings.  Two simple proofs of the feasibility of the linear tracing procedure , 2000 .

[26]  P. Jean-Jacques Herings,et al.  Equilibrium Selection in Stochastic Games , 2001, IGTR.

[27]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.

[28]  T. Parthasarathy,et al.  An orderfield property for stochastic games when one player controls transition probabilities , 1981 .

[29]  Layne T. Watson,et al.  Algorithm 652: HOMPACK: a suite of codes for globally convergent homotopy algorithms , 1987, TOMS.

[30]  R. McKelvey,et al.  Quantal Response Equilibria for Normal Form Games , 1995 .

[31]  Michèle Breton,et al.  Algorithms for Stochastic Games , 1991 .

[32]  Masayuki Takahashi.  Equilibrium points of stochastic non-cooperative $n$-person games , 1964 .

[33]  A. M. Fink,et al.  Equilibrium in a stochastic $n$-person game , 1964 .

[34]  A. Pakes,et al.  Empirical Implications of Alternative Models of Firm Dynamics , 1989 .

[35]  C. E. Lemke,et al.  Equilibrium Points of Bimatrix Games , 1964 .

[36]  A. Pakes,et al.  Stochastic Algorithms, Symmetric Markov Perfect Equilibrium, and the , 2001 .

[37]  K. Judd.  Computational Economics and Economic Theory: Substitutes or Complements , 1997 .

[38]  L. Watson.  A globally convergent algorithm for computing fixed points of C2 maps , 1979 .