A Reinforcement Learning Approach to Solving Incomplete Market Models with Aggregate Uncertainty

We develop a method of solving heterogeneous agent models in which individual decisions depend on the entire cross-sectional distribution of individual state variables, such as incomplete market models with liquidity constraints. Our method is based on the principle of reinforcement learning, and does not require parametric assumptions on either the agents' information set, or on the functional form of the aggregate dynamics.

[1]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[2]  J. L. Hodges,et al.  Discriminatory Analysis - Nonparametric Discrimination: Consistency Properties , 1989 .

[3]  Kenneth L. Judd,et al.  Projection methods for solving aggregate growth models , 1992 .

[4]  A. Kirman.  Whom Or What Does the Representative Individual Represent , 1992 .

[5]  Gerald Tesauro,et al.  TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play , 1994, Neural Computation.

[6]  W. D. Haan,et al.  Heterogeneity, Aggregate Uncertainty, and the Short-Term Interest Rate , 1996 .

[7]  A. Roth,et al.  Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria , 1998 .

[8]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[9]  Per Krusell,et al.  Income and Wealth Heterogeneity in the Macroeconomy , 1998, Journal of Political Economy.

[10]  A. Pakes,et al.  Stochastic Algorithms, Symmetric Markov Perfect Equilibrium, and the , 2001 .

[11]  Felix Kubler,et al.  Computing equilibrium in OLG models with stochastic production , 2004 .

[12]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[13]  Anthony A. Smith,et al.  Advances in Economics and Econometrics: Quantitative Macroeconomic Models with Heterogeneous Agents , 2006 .

[14]  W. D. Haan.  Comparison of Solutions to the Incomplete Markets Model with Aggregate Uncertainty , 2008 .

[15]  Wouter J. den Haan,et al.  Solving the Incomplete Markets Model with Aggregate Uncertainty Using Explicit Aggregation , 2008 .

[16]  Serguei Maliar,et al.  Solving the incomplete markets model with aggregate uncertainty using the Krusell-Smith algorithm , 2010 .

[17]  Panos M. Pardalos,et al.  Approximate dynamic programming: solving the curses of dimensionality , 2009, Optim. Methods Softw..

[18]  Per Krusell,et al.  Quantitative Macroeconomic Models with Heterogeneous Agents , 2009 .

[19]  Warren B. Powell,et al.  An Approximate Dynamic Programming Algorithm for Large-Scale Fleet Management: A Case Application , 2009, Transp. Sci..

[20]  Wouter J. Den Haan,et al.  Computational suite of models with heterogeneous agents: Multi-country real business cycle models , 2011 .

[21]  Solving the incomplete market model with aggregate uncertainty using a perturbation method , 2010 .

[22]  K. Judd,et al.  Computational suite of models with heterogeneous agents: Incomplete markets and aggregate uncertainty , 2010 .

[23]  Eric R. Young.  Solving the incomplete markets model with aggregate uncertainty using the Krusell-Smith algorithm and non-stochastic simulations , 2010 .

[24]  Lilia Maliar,et al.  Solving the Multi-Country Real Business Cycle Model Using Ergodic Set Methods , 2010 .

[25]  Solving the incomplete markets model with aggregate uncertainty using parameterized cross-sectional distributions , 2010 .

[26]  Solving the incomplete markets model with aggregate uncertainty by backward induction , 2010 .

[27]  P. Pichler.  Solving the multi-country real business cycle model using a monomial rule galerkin method , 2011 .

[28]  Warren B. Powell,et al.  Dynamic Programming Models and Algorithms for the Mutual Fund Cash Balance Problem , 2010, Manag. Sci..

[29]  N. Kocherlakota.  Modern macroeconomic models as tools for economic policy , 2010 .

[30]  Bernhard Schölkopf,et al.  Hilbert Space Embeddings and Metrics on Probability Measures , 2009, J. Mach. Learn. Res..

[31]  Wouter J. Den Haan,et al.  Assessing the accuracy of the aggregate law of motion in models with heterogeneous agents , 2010 .

[32]  Warren B. Powell,et al.  “Approximate dynamic programming: Solving the curses of dimensionality” by Warren B. Powell , 2007, Wiley Series in Probability and Statistics.