Numerical Methods for Stochastic Differential Games: The Ergodic Cost Criterion

The Markov chain approximation method is a widely used, relatively easy to apply, and efficient family of methods for the bulk of stochastic control problems in continuous time, for reflected jump-diffusion type models. It is the most general method available, it has been shown to converge under broad conditions, via probabilistic methods, and there are good algorithms for solving the numerical problems, if the dimension is not too high (see the basic reference [16]). We consider a class of stochastic differential games with a reflected diffusion system model and ergodic cost criterion, where the controls for the two players are separated in the dynamics and cost function. The value of the game exists and the numerical method converges to this value as the discretization parameter goes to zero. The actual numerical method solves a stochastic game for a finite state Markov chain and ergodic cost criterion. The essential conditions are nondegeneracy of the diffusion and that a weak local consistency condition hold “almost everywhere” for the numerical approximations, just as for the control problem. The latter is close to a minimal condition. Such ergodic and “separated” game models occur in risk-sensitive and robust control.

[1]  P. Billingsley,et al.  Convergence of Probability Measures , 1969 .

[2]  Ioannis Karatzas,et al.  Brownian Motion and Stochastic Calculus , 1987 .

[3]  P. Dupuis,et al.  On Lipschitz continuity of the solution mapping to the Skorokhod problem , 1991 .

[4]  M. Bardi,et al.  Approximation of differential games of pursuit-evasion by discrete-time games , 1991 .

[5]  M. Falcone,et al.  Fully Discrete Schemes for the Value Function of Pursuit-Evasion Games , 1994 .

[6]  T. Başar,et al.  Advances in Dynamic Games and Applications , 1994 .

[7]  M. Tidball Undiscounted zero sum differential games with stopping times , 1995 .

[8]  G. Olsder New trends in dynamic games and applications , 1995 .

[9]  W. Fleming,et al.  Risk-Sensitive Control on an Infinite Time Horizon , 1995 .

[10]  M. Falcone,et al.  Convergence of Discrete Schemes for Discontinuous Value Functions of Pursuit-Evasion Games , 1995 .

[11]  Odile Pourtallier,et al.  Approximation of the Value Function for a Class of Differential Games with Target , 1996 .

[12]  T. Başar,et al.  Stochastic and differential games : theory and numerical methods , 1999 .

[13]  M. Falcone,et al.  Numerical Methods for Pursuit-Evasion Games via Viscosity Solutions , 1999 .

[14]  H. Kushner Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[15]  H. Kushner Heavy Traffic Analysis of Controlled Queueing and Communication Networks , 2001 .

[16]  HAROLD J. KUSHNER,et al.  Numerical Approximations for Stochastic Differential Games , 2002, SIAM J. Control. Optim..

[17]  Harold J. Kushner The Gauss-Seidel numerical procedure for Markov stochastic games , 2004, IEEE Transactions on Automatic Control.

[18]  Harold J. Kushner Numerical Approximations for Stochastic Differential Games: The Ergodic Case , 2004, SIAM J. Control. Optim..

[19]  S. Ethier,et al.  Markov Processes: Characterization and Convergence , 2005 .