Solving Stochastic Dynamic Programming Problems: A Mixed Complementarity Approach

We present a mixed complementarity problem (MCP) formulation of continuous state dynamic programming problems (DP-MCP). We write the solution to projection methods in value function iteration (VFI) as a joint set of optimality conditions that characterize maximization of the Bellman equation; and approximation of the value function. The MCP approach replaces the iterative component of projection based VFI with a one-shot solution to a square system of complementary conditions. We provide three numerical examples to illustrate our approach.

[1]  T. Rutherford.  Extension of GAMS for complementarity problems arising in applied economic analysis , 1995 .

[2]  Felix Kubler,et al.  Computing equilibrium in OLG models with stochastic production , 2004 .

[3]  Lilia Maliar,et al.  Numerical Methods for Large-Scale Dynamic Economic Models , 2014 .

[4]  Lilia Maliar,et al.  Merging Simulation and Projection Approaches to Solve High-Dimensional Problems , 2012 .

[5]  George Tauchen,et al.  Finite state markov-chain approximations to univariate and vector autoregressions , 1986 .

[6]  L. Mathiesen.  Computation of economic equilibria by a sequence of linear complementarity problems , 1985 .

[7]  Christian P. Traeger.  Why uncertainty matters: discounting under intertemporal risk aversion and ambiguity , 2012, SSRN Electronic Journal.

[8]  Warren B. Powell,et al.  Approximate Dynamic Programming: Solving the Curses of Dimensionality (Wiley Series in Probability and Statistics) , 2007 .

[9]  Tobias N. Rasmussen,et al.  Modeling overlapping generations in a complementarity format , 2004 .

[10]  Che-Lin Su,et al.  Constrained Optimization Approaches to Estimation of Structural Models , 2011 .

[11]  Michael C. Ferris,et al.  Solving equilibrium problems using extended mathematical programming , 2018, Math. Program. Comput..

[12]  K. Judd.  Numerical methods in economics , 1998 .

[13]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[14]  T. Sargent,et al.  Exercises in Dynamic Macroeconomic Theory , 1987 .

[15]  Christian P. Traeger,et al.  Economics of tipping the climate dominoes , 2016 .

[16]  Warren B. Powell,et al.  “Approximate dynamic programming: Solving the curses of dimensionality” by Warren B. Powell , 2007, Wiley Series in Probability and Statistics.

[17]  John Rust.  Numerical dynamic programming in economics , 1996 .

[18]  Derek M Lemoine,et al.  Watch Your Step: Optimal Policy in a Tipping Climate , 2014 .

[19]  Jesús Fernández-Villaverde,et al.  Comparing Solution Methods for Dynamic Equilibrium Economies , 2003 .

[20]  Stephen J. Wright,et al.  Solving Dynamic Programming Problems on a Computational Grid , 2013, Computational Economics.

[21]  M. Ferris,et al.  Complementarity problems in GAMS and the PATH solver 1 This material is based on research supported , 2000 .

[22]  Paul L. Fackler,et al.  Applied Computational Economics and Finance , 2002 .

[23]  Kenneth L. Judd,et al.  Advances in Numerical Dynamic Programming and New Applications , 2014 .

[24]  Jeremy T. Fox,et al.  Improving the Numerical Performance of Static and Dynamic Aggregate Discrete Choice Random Coefficients Demand Estimation by Jean-pierre Dubé, , 2012 .

[25]  Michael C. Ferris,et al.  An extended mathematical programming framework , 2009, Comput. Chem. Eng..

[26]  Thomas F. Rutherford,et al.  Approximating infinite-horizon models in a complementarity format: A primer in dynamic general equilibrium analysis , 2002 .

[27]  Derek M Lemoine,et al.  Managing Climate Change Under Uncertainty: Recursive Integrated Assessment at an Inflection Point , 2017 .

[28]  I. Rudik.  Optimal Climate Policy When Damages are Unknown , 2019, American Economic Journal: Economic Policy.

[29]  S. Dirkse,et al.  The path solver: a nommonotone stabilization scheme for mixed complementarity problems , 1995 .

[30]  Jesús Fernández-Villaverde,et al.  A Comparison of Programming Languages in Economics , 2014 .

[31]  Kenneth L. Judd,et al.  DSICE: A Dynamic Stochastic Integrated Model of Climate and Economy , 2012, SSRN Electronic Journal.

[32]  Nancy L. Stokey,et al.  Recursive methods in economic dynamics , 1989 .

[33]  Pablo A. Guerrón-Quintana,et al.  Nonlinear Adventures at the Zero Lower Bound , 2012 .

[34]  Lilia Maliar,et al.  Smolyak Method for Solving Dynamic Economic Models: Lagrange Interpolation, Anisotropic Grid and Adaptive Domain , 2013 .

[35]  Christian P. Traeger,et al.  A 4-Stated DICE: Quantitatively Addressing Uncertainty Effects in Climate Change , 2013, Environmental and Resource Economics.

[36]  Kenneth L. Judd,et al.  A Nonlinear Certainty Equivalent Approximation Method for Dynamic Stochastic Problems , 2015 .