Solving nonlinear dynamic models by iterative dynamic programming

The paper presents a discrete-time dynamic programming algorithm that is suitable to track nonlinearities in intertemporal optimization problems. In contrast to using linearization methods for solving intertemporal models, the proposed algorithm operates globally by iteratively computing the value function and the controls in feedback form on a grid. A conjecture of how the trajectories might behave is analytically obtained by letting the discount rate approach infinity. The dynamic found serves as a useful device for computing the trajectories for finite discount rates employing the algorithm. Commencing with a large step and grid size, and then pursuing time step and grid refinements allows for the replication of the nonlinear dynamics for various finite discount rates. As the time step and grid size shrink, the discretization errors vanish. The algorithm is applied to three economic examples. Two examples are of deterministic type; the third is stochastic. In the deterministic cases limit cycles are detected.

[1]  Colin W. Clark,et al.  Economically optimal policies for the utilization of biologically renewable resources , 1971 .

[2]  L. Berkovitz.  Optimal Control Theory , 1974 .

[3]  Colin W. Clark,et al.  Mathematical Bioeconomics: The Optimal Management of Renewable Resources. , 1993 .

[4]  K. Lommatzsch.  Berkovitz, L. D., Optimal Control Theory. Berlin‐Heidelberg‐New York. Springer‐Verlag. 1974. 304 S., 10 Abb., DM 23,30 (Appl. Mathematical Sciences 12) , 1978 .

[5]  F. Clarke,et al.  The Optimal Exploitation of Renewable Resource Stocks: Problems of Irreversible Investment , 1979 .

[6]  Tapan Mitra,et al.  Intertemporal allocation with a non-convex technology: The aggregative framework , 1982 .

[7]  Kazuo Nishimura,et al.  A Complete Characterization of Optimal Growth Paths in an Aggregated Model with a Non-Concave Production Function , 1983 .

[8]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[9]  William J. Reed,et al.  The effects of the risk of fire on the optimal rotation of a forest , 1984 .

[10]  L. Montrucchio,et al.  On the indeterminacy of capital accumulation paths , 1986 .

[11]  W. Brock.  Distinguishing random and deterministic systems: Abridged version , 1986 .

[12]  M. Falcone.  A numerical approach to the infinite horizon problem of deterministic control theory , 1987 .

[13]  R. Lucas.  On the Mechanics of Economic Development , 1988 .

[14]  Robert E. Lucas,et al.  ON THE MECHANICS OF ECONOMIC GROWTH , 1988 .

[15]  Michele Boldrin,et al.  Equilibrium models displaying endogenous fluctuations and chaos: A survey , 1988 .

[16]  P. Romer.  Endogenous Technological Change , 1989, Journal of Political Economy.

[17]  Harald Uhlig,et al.  Solving Nonlinear Stochastic Growth Models: a Comparison of Alternative Solution Methods , 1989 .

[18]  M. Paul,et al.  ENDOGENOUS TECHNICAL CHANGE , 1990 .

[19]  R. Luus.  Optimal control by dynamic programming using systematic reduction in grid size , 1990 .

[20]  Michele Boldrin,et al.  Sources of Complex Dynamics in Two-Sector Growth Models , 2021, Cycles and Chaos in Economic Equilibrium.

[21]  G. Sorger.  On the optimality of given feedback controls , 1990 .

[22]  Maurizio Falcone,et al.  Corrigenda: A numerical approach to the infinite horizon problem of deterministic control theory , 1991 .

[23]  Douglas M. Patterson,et al.  Nonlinear Dynamics, Chaos, And Instability , 1994 .

[24]  Gustav Feichtinger,et al.  On the optimality of limit cycles in dynamic economic systems , 1991 .

[25]  J. Danthine,et al.  Computing Equilibria of Non-Optimal Economies , 1993 .

[26]  William A. Brock,et al.  PATHWAYS TO RANDOMNESS IN THE ECONOMY: EMERGENT NONLINEARITY AND CHAOS IN ECONOMICS AND FINANCE , 1993 .

[27]  On the optimal exploitation of interacting resources , 1994 .

[28]  W. Semmler.  Business cycles : theory and empirical methods , 1994 .

[29]  W. Semmler,et al.  Growth and finance: An intertemporal model , 1995 .

[30]  Celso Grebogi,et al.  Shadowing in Chaotic Systems , 1995 .

[31]  Gang Gong,et al.  Estimating parameters of real business cycle models , 1996 .

[32]  Luigi Montrucchio,et al.  Dynamical Systems that Solve Continuous-Time Concave Optimization Problems: , 2021, Cycles and Chaos in Economic Equilibrium.