Numerical Methods for Stochastic Differential Games: The Ergodic Cost Criterion

The Markov chain approximation method is a widely used, relatively easy to apply, and efficient family of methods for the bulk of stochastic control problems in continuous time, for reflected jump-diffusion type models. It is the most general method available, it has been shown to converge under broad conditions, via probabilistic methods, and there are good algorithms for solving the numerical problems, if the dimension is not too high (see the basic reference [16]). We consider a class of stochastic differential games with a reflected diffusion system model and ergodic cost criterion, where the controls for the two players are separated in the dynamics and cost function. The value of the game exists and the numerical method converges to this value as the discretization parameter goes to zero. The actual numerical method solves a stochastic game for a finite state Markov chain and ergodic cost criterion. The essential conditions are nondegeneracy of the diffusion and that a weak local consistency condition hold “almost everywhere” for the numerical approximations, just as for the control problem. The latter is close to a minimal condition. Such ergodic and “separated” game models occur in risk-sensitive and robust control.