Extremal Shift Rule for Continuous-Time Zero-Sum Markov Games

In the paper we consider the controlled continuous-time Markov chain describing the interacting particles system with the finite number of types. The system is controlled by two players with the opposite purposes. This Markov game converges to a zero-sum differential game when the number of particles tends to infinity. Krasovskii–Subbotin extremal shift provides the optimal strategy in the limiting game. The main result of the paper is the near optimality of the Krasovskii–Subbotin extremal shift rule for the original Markov game.

[1]  M. Benaïm,et al.  A class of mean field interaction models for computer and communication systems , 2008, 2008 6th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks and Workshops.

[2]  Vassili N. Kolokoltsov.  Nonlinear Markov Processes and Kinetic Equations: Preface , 2010 .

[3]  Abraham Neyman,et al.  Continuous-time stochastic games , 2017, Games Econ. Behav..

[4]  Y. Averboukh.  Universal Nash Equilibrium Strategies for Differential Games , 2013, 1306.2297.

[5]  J. Norris,et al.  Differential equation approximations for Markov chains , 2007, 0710.3269.

[6]  V. Kolokoltsov.  Nonlinear Markov Games on a Finite State Space (Mean-field and Binary Interactions) , 2012 .

[7]  N. Krasovskii,et al.  Stochastic guide for a time-delay object in a positional differential game , 2012 .

[8]  N. Krasovskii,et al.  Unification of differential games, generalized solutions of the Hamilton-Jacobi equations, and a stochastic guide , 2009 .

[9]  Yehuda Levy.  Continuous-Time Stochastic Games of Fixed Duration , 2013, Dyn. Games Appl..

[10]  J. Boudec,et al.  A class of mean field interaction models for computer and communication systems , 2008, Perform. Evaluation.

[11]  Bruno Gaujal,et al.  Mean Field for Markov Decision Processes: From Discrete to Continuous Optimization , 2010, IEEE Transactions on Automatic Control.

[12]  M. Bardi,et al.  Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations , 1997 .

[13]  A. I. Subbotin,et al.  Game-Theoretical Control Problems , 1987 .

[14]  A. I. Subbotin.  Generalized Solutions of First Order PDEs , 1995 .

[15]  van der J Jan Wal,et al.  On Markov games , 1976 .

[16]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[17]  N. Krasovskii,et al.  An approach-evasion differential game: Stochastic guide , 2010 .

[18]  N. Krasovskii,et al.  On a differential interception game , 2010 .