An optimal one-way multigrid algorithm for discrete-time stochastic control

The numerical solution of discrete-time stationary infinite-horizon discounted stochastic control problems is considered for the case where the state space is continuous and the problem is to be solved approximately, within a desired accuracy. After a discussion of problem discretization, the authors introduce a multigrid version of the successive approximation algorithm that proceeds 'one way' from coarse to fine grids, and analyze its computational requirements as a function of the desired accuracy and of the discount factor. They also study the effects of a certain mixing (ergodicity) condition on the algorithm's performance. It is shown that the one-way multigrid algorithm improves upon the complexity of its single-grid variant and is, in a certain sense, optimal. >

[1]  V. Smirnov.  Integration and functional analysis , 1964 .

[2]  D. Blackwell.  Discounted Dynamic Programming , 1965 .

[3]  E. Denardo.  CONTRACTION MAPPINGS IN THE THEORY UNDERLYING DYNAMIC PROGRAMMING , 1967 .

[4]  R. A. Silverman,et al.  Introductory Real Analysis , 1972 .

[5]  R. Ash.  Measure, integration, and functional analysis , 1971 .

[6]  D. Bertsekas.  Convergence of discretization procedures in dynamic programming , 1975 .

[7]  P. Schweitzer.  Contraction mappings underlying undiscounted Markov decision problems—II , 1978 .

[8]  Ward Whitt,et al.  Approximations of Dynamic Programs, I , 1978, Math. Oper. Res..

[9]  C. Douglas.  Multi-Grid Algorithms with Applications to Elliptic Boundary Value Problems , 1984 .

[10]  A. Werschulz.  What is the Complexity of the Fredholm Problem of the Second Kind , 1984 .

[11]  Wolfgang Hackbusch,et al.  Multi-grid methods and applications , 1985, Springer series in computational mathematics.

[12]  P. L’Ecuyer,et al.  Approximation and bounds in discrete event dynamic programming , 1986 .

[13]  D. Bertsekas,et al.  Adaptive aggregation methods for discounted dynamic programming , 1986, 1986 25th IEEE Conference on Decision and Control.

[14]  N. McKay,et al.  A dynamic programming approach to trajectory planning of robotic manipulators , 1986 .

[15]  R. Hoppe.  Multi-grid methods for Hamilton-Jacobi-Bellman equations , 1986 .

[16]  J. Tsitsiklis,et al.  Intractable problems in control theory , 1986 .

[17]  John N. Tsitsiklis,et al.  The Complexity of Markov Decision Processes , 1987, Math. Oper. Res..

[18]  Jean-Philippe Chancelier,et al.  Dynamic programming complexity and application , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[19]  P. L’Ecuyer.  Computing Approximate Solutions to Markov Renewal Programs with Continuous State Spaces , 1989 .

[20]  Chows Chee-Seng.  Multigrid algorithms and complexity results for discrete-time stochastic control and related fixed-point problems , 1989 .

[21]  John N. Tsitsiklis,et al.  The complexity of dynamic programming , 1989, J. Complex..

[22]  O. Hernández-Lerma.  Adaptive Markov Control Processes , 1989 .