Numerical methods for controls for nonlinear stochastic systems with delays and jumps: applications to admission control

We consider numerical methods of the Markov chain approximation type for computing optimal controls and value functions for systems governed by nonlinear stochastic delay equations. Earlier work did not allow Poisson random measure driving processes or delays that are concentrated on points with positive probability. In addition, the Poisson measures can be controlled. Previous proofs are not adequate for the present case. The algorithms are developed and convergence proved as the approximating parameters go to their limits. One motivating example concerns admissions control to a network, where the file arrival process is governed by a Poisson process, and arrivals might be admitted or not, according to the control, which leads to a controlled Poisson process. Numerical data for such an example are presented. The original problem is recast in terms of a transportation equation, which allows the development of practical algorithms. For the problems of interest, alternative methods can entail prohibitive memory and computational requirements.