The Linear Programming Approach to Solving Large Scale Dynamic Stochastic Games

In this paper we introduce a new method to approximate Markov perfect equilibrium in large scale dynamic stochastic games that are not amenable to exact solution due to the curse of dimensionality. The method is based on an algorithm that iterates an approximate best response operator computed via the ‘approximate linear programming’ approach. We provide results that lend theoretical support to our approximation. We test our method on a class of dynamic models of imperfect competition. Our results suggest that the approach we propose significantly expands the set of models that can be analyzed computationally. This substantially enhances the applicability of dynamic oligopoly models among other applications of dynamic stochastic games.

[1]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.

[2]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Vol. II , 1976 .

[3]  T. Başar,et al.  Dynamic Noncooperative Game Theory , 1982 .

[4]  E. Maskin,et al.  Overview and quantity competition with large fixed costs , 1988 .

[5]  E. Maskin,et al.  A theory of dynamic oligopoly. , 1989 .

[6]  Andrew Caplin,et al.  Aggregation and Imperfect Competition: On the Existence of Equilibrium , 1991 .

[7]  Jerzy A. Filar,et al.  Nonlinear programming and stationary equilibria in stochastic games , 1991, Math. Program..

[8]  Drew Fudenberg,et al.  Game theory (3. pr.) , 1991 .

[9]  Michèle Breton,et al.  Algorithms for Stochastic Games , 1991 .

[10]  A. Pakes,et al.  Computing Markov Perfect Nash Equilibria: Numerical Implications of a Dynamic Differentiated Product Model , 1992 .

[11]  Michael A. Trick,et al.  A Linear Programming Approach to Solving Stochastic Dynamic Programming , 1993 .

[12]  A. Pakes,et al.  Markov-Perfect Industry Dynamics: A Framework for Empirical Work , 1995 .

[13]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[14]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[15]  J. Filar,et al.  Competitive Markov Decision Processes , 1996 .

[16]  Autam,et al.  Dynamic Equilibrium in the Hospital Industry , 1997 .

[17]  Stanley E. Zin,et al.  SPLINE APPROXIMATIONS TO VALUE FUNCTIONS: Linear Programming Approach , 1997 .

[18]  Per Krusell,et al.  Income and Wealth Heterogeneity in the Macroeconomy , 1998, Journal of Political Economy.

[19]  K. Judd.  Numerical methods in economics , 1998 .

[20]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[21]  P. Herings,et al.  Stationary Equilibria in Stochastic Games: Structure, Selection and Computation , 2000 .

[22]  A. Pakes,et al.  Stochastic Algorithms, Symmetric Markov Perfect Equilibrium, and the , 2001 .

[23]  Benjamin Van Roy,et al.  The Linear Programming Approach to Approximate Dynamic Programming , 2003, Oper. Res..

[24]  U. Doraszelski.  An R&D Race with Knowledge Accumulation , 2003 .

[25]  Rosa L. Matzkin,et al.  The Browser War - Econometric Analysis of Markov Perfect Equilibrium in Markets with Network Effects 1 , 2004 .

[26]  Benjamin Van Roy,et al.  On Constraint Sampling in the Linear Programming Approach to Approximate Dynamic Programming , 2004, Math. Oper. Res..

[27]  Benjamin Van Roy,et al.  Markov Perfect Industry Dynamics with Many Firms , 2005 .

[28]  Stephen P. Ryan.  The Costs of Environmental Regulation in a Concentrated Industry , 2012 .

[29]  M. Satterthwaite,et al.  Computable Markov-Perfect Industry Dynamics: Existence, Purification, and Multiplicity , 2007 .

[30]  Ariel Pakes,et al.  Chapter 30 A Framework for Applied Dynamic Analysis in IO , 2007 .

[31]  Srihari Govindan,et al.  Global Newton Method for Stochastic Games , 2009 .

[32]  Robert Wilson,et al.  Global Newton Method for stochastic games , 2009, J. Econ. Theory.

[33]  Ulrich Doraszelski,et al.  A User's Guide to Solving Dynamic Stochastic Games Using the Homotopy Method , 2008, Oper. Res..

[34]  Benjamin Van Roy,et al.  Computational Methods for Oblivious Equilibrium , 2010, Oper. Res..