Numerical Approximations for Stochastic Differential Games: The Ergodic Case

The Markov chain approximation method is a widely used, relatively easy to use, and efficient family of methods for the bulk of stochastic control problems in continuous time for reflected-jump-diffusion-type models. It has been shown to converge under broad conditions, and there are good algorithms for solving the numerical problems if the dimension is not too high. We consider a class of stochastic differential games with a reflected diffusion system model and ergodic cost criterion and where the controls for the two players are separated in the dynamics and cost function. It is shown that the value of the game exists and that the numerical method converges to this value as the discretization parameter goes to zero. The actual numerical method solves a stochastic game for a finite state Markov chain and ergodic cost criterion. The essential conditions are nondegeneracy and that a weak local consistency condition hold "almost everywhere" for the numerical approximations, just as for the control problem.

[1]  W. Fleming,et al.  Risk-Sensitive Control on an Infinite Time Horizon , 1995 .

[2]  J. Schwartz,et al.  Linear Operators. Part I: General Theory. , 1960 .

[3]  M. Falcone,et al.  Fully Discrete Schemes for the Value Function of Pursuit-Evasion Games , 1994 .

[4]  Martin I. Reiman,et al.  Open Queueing Networks in Heavy Traffic , 1984, Math. Oper. Res..

[5]  HAROLD J. KUSHNER,et al.  Numerical Approximations for Stochastic Differential Games , 2002, SIAM J. Control. Optim..

[6]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[7]  H. Kushner,et al.  Admission Control for Combined Guaranteed Performance and Best Effort Communications Systems Under Heavy Traffic , 1999 .

[8]  S. Ethier,et al.  Markov Processes: Characterization and Convergence , 2005 .

[9]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[10]  Vivek S. Borkar,et al.  Optimal Control of Diffusion Processes , 1989 .

[11]  P. Billingsley,et al.  Convergence of Probability Measures , 1969 .

[12]  Ruth J. Williams,et al.  A boundary property of semimartingale reflecting Brownian motions , 1988 .

[13]  M. Falcone,et al.  Convergence of Discrete Schemes for Discontinuous Value Functions of Pursuit-Evasion Games , 1995 .

[14]  F. Smithies.  Linear Operators , 2019, Nature.

[15]  Pushkin Kachroo,et al.  Robust Feedback Control of a Single Server Queueing System , 1999, Math. Control. Signals Syst..

[16]  Pushkin Kachroo,et al.  Robust L2-gain control for nonlinear systems with projection dynamics and input constraints: an example from traffic control , 1999, Autom..

[17]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[18]  Ruth J. Williams,et al.  Brownian Models of Open Queueing Networks with Homogeneous Customer Populations , 1987 .

[19]  M. Tidball.  Undiscounted zero sum differential games with stopping times , 1995 .

[20]  H. Kushner.  Heavy Traffic Analysis of Controlled Queueing and Communication Networks , 2001 .

[21]  M. Falcone,et al.  Numerical Methods for Pursuit-Evasion Games via Viscosity Solutions , 1999 .

[22]  Mabel M. Tidball,et al.  Zero Sum Differential Games With Stopping Times: Some Results About its Numerical Resolution , 1994 .

[23]  H. Kushner.  Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[24]  P. Dupuis,et al.  On Lipschitz continuity of the solution mapping to the Skorokhod problem , 1991 .

[25]  Odile Pourtallier,et al.  Approximation of the Value Function for a Class of Differential Games with Target , 1996 .