Nonzero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates

This paper attempts to study two-person nonzero-sum games for denumerable continuous-time Markov chains determined by transition rates, with an expected average criterion. The transition rates are allowed to be unbounded, and the payoff functions may be unbounded from above and from below. We give suitable conditions under which the existence of a Nash equilibrium is ensured. More precisely, using the socalled “vanishing discount” approach, a Nash equilibrium for the average criterion is obtained as a limit point of a sequence of equilibrium strategies for the discounted criterion as the discount factors tend to zero. Our results are illustrated with a birth-and-death game.

[1]  S. Meyn,et al.  Computable exponential convergence rates for stochastically ordered Markov processes , 1996 .

[2]  A. Nowak.  Some remarks on equilibria in semi-Markov games , 2000 .

[3]  P. Kakumanu,et al.  Nondiscounted Continuous Time Markovian Decision Process with Countable State Space , 1972 .

[4]  K. Chung,et al.  Lectures on Boundary Theory for Markov Chains. , 1970 .

[5]  I. Gihman,et al.  Controlled Stochastic Processes , 1979 .

[6]  Kensuke Tanaka,et al.  On Continuous Time Markov Games With The Expected Average Reward Criterion , 1977 .

[7]  L. Rogers,et al.  Diffusions, Markov processes, and martingales , 1979 .

[8]  O. Hernández-Lerma,et al.  Continuous-time controlled Markov chains , 2003 .

[9]  M. Bartlett,et al.  Markov Processes and Potential Theory , 1972, The Mathematical Gazette.

[10]  Kensuke Tanaka,et al.  A noncooperativen-person semi-Markov game with a separable metric state space , 1984 .

[11]  Bharat T. Doshi,et al.  Continuous time control of Markov processes on an arbitrary state space: Average return criterion , 1976 .

[12]  K. Chung.  Markov Chains with Stationary Transition Probabilities , 1961 .

[13]  Xianping Guo,et al.  A note on optimality conditions for continuous-time Markov decision processes with average cost criterion , 2001, IEEE Trans. Autom. Control..

[14]  T. Rolski.  Stochastic Processes for Insurance and Finance , 1999 .

[15]  Xianping Guo,et al.  Drift and monotonicity conditions for continuous-time controlled Markov chains with an average criterion , 2003, IEEE Trans. Autom. Control..

[16]  K. Ramachandran.  A convergence method for stochastic differential games with a small parameter , 1999 .

[17]  Sagnik Sinha,et al.  Zero-sum two-person semi-Markov games , 1992, Journal of Applied Probability.

[18]  M. Rao.  Stochastic Processes: General Theory , 1995 .

[19]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[20]  A. Hordijk,et al.  Contraction Conditions for Average and α-Discount Optimality in Countable State Markov Games with Unbounded Rewards , 1997, Math. Oper. Res..

[21]  Onésimo Hernández-Lerma,et al.  Zero-Sum Stochastic Games in Borel Spaces: Average Payoff Criteria , 2000, SIAM J. Control. Optim..

[22]  W. Polowczuk.  Nonzero-sum semi-Markov games with countable state spaces , 2000 .

[23]  W. J. Anderson.  Continuous-Time Markov Chains , 1991 .

[24]  O. Hernández-Lerma,et al.  Further topics on discrete-time Markov control processes , 1999 .

[25]  W. Feller,et al.  On the integro-differential equations of purely discontinuous Markoff processes , 1940 .

[26]  J. Filar,et al.  Competitive Markov Decision Processes , 1996 .

[27]  Vivek S. Borkar,et al.  Denumerable state stochastic games with limiting average payoff , 1993 .

[28]  On an N-person noncooperative Markov game with a metric state space , 1984 .

[29]  M. K. Ghosh,et al.  Stochastic Games with Average Payoff Criterion , 1998 .