Convergence of Numerical Method for Multistate Stochastic Dynamic Programming

Abstract Convergence of corrections is examined for a predictor-corrector method to solve Bellman equations of multi-state stochastic optimal control in continuous time. Quadratic costs and constrained control are assumed. A heuristically linearized comparison equation makes the nonlinear, discontinuous Bellman equation amenable to linear convergence analysis. Convergence is studied using the Fourier stability method. A uniform mesh ratio type condition for the convergence is results. The results are valid for both Gaussian and Poisson type stochastic noise. The convergence criteria has been extremely useful for solving the larger multi-state problems on vector supercomputers and massively parallel processors