Numerical Approximations for Stochastic Differential Games

The Markov chain approximation method is a widely used, robust, relatively easy to use, and efficient family of methods for the bulk of stochastic control problems in continuous time for reflected-jump-diffusion-type models. It has been shown to converge under broad conditions, and there are good algorithms for solving the numerical problems if the dimension is not too high. Versions of these methods have been used in applications to various two-player differential and stochastic dynamic games for a long time, and proofs of convergence are available for some cases, mainly using PDE-type techniques. In this paper, purely probabilistic proofs of convergence are given for a broad class of such problems, where the controls for the two players are separated in the dynamics and cost function, and which cover a substantial class not dealt with in previous works. Discounted and stopping time cost functions are considered. Finite horizon problems and problems where the process is stopped on first hitting an a priori given boundary can be dealt with by adapting the methods of [H. J. Kushner and P. Dupuis, Numerical Methods for Stochastic Control Problems, in Continuous Time, 2nd ed., Springer-Verlag, Berlin, New York, 2001] as done in this paper for the treated problems. The essential conditions are the weak-sense existence and uniqueness of solutions, an "almost everywhere" continuity condition, and that a weak local consistency condition holds "almost everywhere" for the numerical approximations, just as for the control problem. There are extensions to problems with controlled variance and jumps.

[1]  M. Falcone,et al.  Numerical Methods for Pursuit-Evasion Games via Viscosity Solutions , 1999 .

[2]  W. Fleming.  The convergence problem for differential games , 1961 .

[3]  Mabel M. Tidball,et al.  Zero Sum Differential Games With Stopping Times: Some Results About its Numerical Resolution , 1994 .

[4]  M. Falcone,et al.  Convergence of Discrete Schemes for Discontinuous Value Functions of Pursuit-Evasion Games , 1995 .

[5]  T. Kurtz.  Approximation of Population Processes , 1987 .

[6]  Pushkin Kachroo,et al.  Robust Feedback Control of a Single Server Queueing System , 1999, Math. Control. Signals Syst..

[7]  T. E. S. Raghavan,et al.  Algorithms for stochastic games — A survey , 1991, ZOR Methods Model. Oper. Res..

[8]  Harold J. Kushner,et al.  On stochastic differential games: Sufficient conditions that a given strategy be a saddle point, and numerical procedures for the solution of the game☆ , 1969 .

[9]  Tamer Basar,et al.  H∞-Optimal Control and Related , 1991 .

[10]  Ioannis Karatzas,et al.  Brownian Motion and Stochastic Calculus , 1987 .

[11]  Pushkin Kachroo,et al.  Robust L2-gain control for nonlinear systems with projection dynamics and input constraints: an example from traffic control , 1999, Autom..

[12]  H. Kushner.  Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[13]  P. Dupuis,et al.  On Lipschitz continuity of the solution mapping to the Skorokhod problem , 1991 .

[14]  William M. McEneaney,et al.  Risk-Sensitive and Robust Escape Criteria , 1997 .

[15]  M. Falcone,et al.  Fully Discrete Schemes for the Value Function of Pursuit-Evasion Games , 1994 .

[16]  J. Harrison,et al.  Reflected Brownian Motion on an Orthant , 1981 .

[17]  Odile Pourtallier,et al.  Approximation of the Value Function for a Class of Differential Games with Target , 1996 .

[18]  H. Kushner,et al.  Finite state stochastic games: Existence theorems and computational procedures , 1969 .

[19]  Martin I. Reiman,et al.  Open Queueing Networks in Heavy Traffic , 1984, Math. Oper. Res..

[20]  Pierpaolo Soravia.  ${\cal H}_\infty$ Control of Nonlinear Systems: Differential Games and Viscosity Solutions , 1996 .

[21]  H. Kushner,et al.  Admission Control for Combined Guaranteed Performance and Best Effort Communications Systems Under Heavy Traffic , 1999 .

[22]  Harold J. Kushner.  Jump-Diffusions with Controlled Jumps: Existence and Numerical Methods , 2000 .

[23]  R. Elliott,et al.  The Existence Of Value In Differential Games , 1972 .

[24]  Ruth J. Williams,et al.  A boundary property of semimartingale reflecting Brownian motions , 1988 .

[25]  M. Tidball.  Undiscounted zero sum differential games with stopping times , 1995 .

[26]  H. Kushner.  Heavy Traffic Analysis of Controlled Queueing and Communication Networks , 2001 .

[27]  Ruth J. Williams,et al.  Brownian Models of Open Queueing Networks with Homogeneous Customer Populations , 1987 .

[28]  J. Neveu,et al.  Mathematical foundations of the calculus of probability , 1965 .

[29]  M. Bardi,et al.  Approximation of differential games of pursuit-evasion by discrete-time games , 1991 .

[30]  W. Fleming,et al.  Risk-Sensitive Control on an Infinite Time Horizon , 1995 .

[31]  Harold J. Kushner.  Numerical Approximations for Stochastic Differential Games: The Ergodic Case , 2004, SIAM J. Control. Optim..

[32]  Wendell H. Fleming,et al.  12. The Convergence Problem for Differential Games, II , 1964 .

[33]  V. Benes.  Existence of Optimal Stochastic Control Laws , 1971 .