Using Randomization to Break the Curse of Dimensionality

This paper introduces random versions of successive approximations and multigrid algorithms for computing approximate solutions to a class of finite and infinite horizon Markovian decision problems. The author proves that these algorithms succeed in breaking the 'curse of dimensionality' for a subclass of Markovian decision problems known as discrete decision processes.

[1]  K. F. Roth.  On irregularities of distribution , 1954 .

[2]  R. Bellman,et al.  Polynomial approximation—a new computational technique in dynamic programming: Allocation processes , 1962 .

[3]  Stuart E. Dreyfus,et al.  Applied Dynamic Programming , 1965 .

[4]  J. Lamperti.  ON CONVERGENCE OF STOCHASTIC PROCESSES , 1962 .

[5]  D. Blackwell.  Discounted Dynamic Programming , 1965 .

[6]  R. Bellman.  Dynamic programming. , 1957, Science.

[7]  L. B. Rall,et al.  Computational Solution of Nonlinear Operator Equations , 1969 .

[8]  B. Fox.  Discretizing dynamic programs , 1973 .

[9]  Michael B. Marcus,et al.  Central limit theorems for C(S)-valued random variables , 1975 .

[10]  D. Bertsekas.  Convergence of discretization procedures in dynamic programming , 1975 .

[11]  James W. Daniel,et al.  Splines and efficiency in dynamic programming , 1976 .

[12]  Martin L. Puterman,et al.  On the Convergence of Policy Iteration in Stationary Dynamic Programming , 1979, Math. Oper. Res..

[13]  Henryk Wozniakowski,et al.  A general theory of optimal algorithms , 1980, ACM monograph series.

[14]  R. Graham,et al.  On irregularities of distribution of real sequences. , 1981, Proceedings of the National Academy of Sciences of the United States of America.

[15]  P. Anselone,et al.  A unified framework for the discretization of nonlinear operator equations , 1981 .

[16]  A. Skorokhod.  Random Linear Operators , 1983 .

[17]  John Rust.  Structural estimation of markov decision processes , 1986 .

[18]  Henryk Wozniakowski,et al.  Information-based complexity , 1987, Nature.

[19]  J. Wellner,et al.  Empirical Processes with Applications to Statistics , 2009 .

[20]  John Rust.  Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher , 1987 .

[21]  J. Tsitsiklis,et al.  An optimal multigrid algorithm for continuous state discrete time stochastic control , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[22]  John Rust.  Maximum likelihood estimation of discrete control processes , 1988 .

[23]  H. Woxniakowski.  Information-Based Complexity , 1988 .

[24]  D. Pollard.  Asymptotics via Empirical Processes , 1989 .

[25]  John N. Tsitsiklis,et al.  The complexity of dynamic programming , 1989, J. Complex..

[26]  George Tauchen,et al.  Quadrature-Based Methods for Obtaining Approximate Solutions to Nonlinear Asset Pricing Models , 1991 .

[27]  H. Wozniakowski.  Average case complexity of multivariate integration , 1991 .

[28]  H. Wozniakowski.  Average case complexity of linear multivariate problems , 1993, math/9307234.

[29]  Henryk Wozniakowski,et al.  The Monte Carlo Algorithm With a Pseudorandom Generator , 1992 .

[30]  Grzegorz W. Wasilkowski,et al.  The Computational Complexity of Differential and Integral Equations: An Information-Based Approach (Arthur G. Werschulz) , 1994, SIAM Rev..

[31]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[32]  Ellen R. McGrattan,et al.  Mechanics of forming and estimating dynamic linear economies , 1994 .

[33]  John N. Tsitsiklis,et al.  Asynchronous stochastic approximation and Q-learning , 1994, Mach. Learn..

[34]  Kenneth I. Wolpin,et al.  The Solution and Estimation of Discrete Choice Dynamic Programming Models by Simulation and Interpol , 1994 .

[35]  Manuel S. Santos,et al.  Accuracy Estimates for a Numerical Approach to Stochastic Growth Models , 1995 .

[36]  Kenneth L. Judd,et al.  Approximation, perturbation, and projection methods in economic analysis , 1996 .

[37]  John Rust.  Numerical dynamic programming in economics , 1996 .

[38]  Stanley J. Rosenschein,et al.  Learning to act using real-time dynamic programming , 1996 .