Dynamic Programming Deconstructed

A common strategy in economic modeling is to try to manipulate Bellman equations into advantageous forms without breaking their link to optimality. In this paper we provide a theoretical framework and a set of results that transform this art into a science. In particular, we (a) clarify the link between such manipulations and optimality of the resulting policies, including when this link breaks down, (b) establish a connection between contractivity of the modified Bellman operators and Bellman's principle of optimality, (c) use manipulations of the Bellman equation to extend the set of algorithms for obtaining optimal policies, (d) extend the set of models that can be transformed to include recursive preferences and other forms of nonseparability, (e) find new applications of these manipulations and use them to obtain new results, and (f) use these methods to simplify the Bellman equation in a range of recent applications.

[1]  Per Krusell,et al.  Income and Wealth Heterogeneity in the Macroeconomy , 1998, Journal of Political Economy.

[2]  Igor Livshits,et al.  Consumer Bankruptcy: A Fresh Start , 2007 .

[3]  R. Bidder,et al.  Robust Animal Spirits , 2012 .

[4]  J. McCall.  Economics of Information and Job Search , 1970 .

[5]  Anna Jaskiewicz,et al.  Stochastic optimal growth model with risk sensitive preferences , 2015, J. Econ. Theory.

[6]  A. Norets,et al.  Continuity and differentiability of expected value functions in dynamic discrete choice models , 2010 .

[7]  Thomas D. Tallarini.  Risk-sensitive real business cycles☆ , 2000 .

[8]  Massimo Marinacci,et al.  Recursive smooth ambiguity preferences , 2009, J. Econ. Theory.

[9]  Patrick Kofod Mogensen.  Solving Dynamic Discrete Choice Models: Integrated or Expected Value Function? , 2018 .

[10]  John Rust.  Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher , 1987 .

[11]  V. Martins-da-Rocha,et al.  Existence and uniqueness of a fixed-point for local contractions , 2010 .

[12]  Thomas Winberry,et al.  A method for solving and estimating heterogeneous agent macro models , 2018 .

[13]  Chong Wang,et al.  Optimal Consumption and Savings with Stochastic Income and Recursive Utility , 2013, J. Econ. Theory.

[14]  Victor Aguirregabiria,et al.  Swapping the Nested Fixed-Point Algorithm: a Class of Estimators for Discrete Markov Decision Models , 2002 .

[15]  S. Pashchenko.  Accounting for Non-Annuitization , 2010 .

[16]  J. P. Rincón-Zapatero,et al.  Existence and Uniqueness of Solutions to the Bellman Equation in the Unbounded Case , 2003 .

[17]  A. McLennan,et al.  Some People Never Learn , Rationally , 2013 .

[18]  Kartik B. Athreya.  Default, insurance, and debt over the life-cycle , 2008 .

[19]  P. Weil.  Nonexpected Utility in Macroeconomics , 1990 .

[20]  R. Reis,et al.  Targeted Transfers and the Fiscal Response to the Great Recession , 2011 .