Nonzero-Sum Risk-Sensitive Stochastic Games on a Countable State Space

The infinite horizon risk-sensitive discounted-cost and ergodic-cost nonzero-sum stochastic games for controlled Markov chains with countably many states are analyzed. For the discounted-cost game, we prove the existence of Nash equilibrium strategies in the class of Markov strategies under fairly general conditions. Under an additional weak geometric ergodicity condition and a small cost criterion, the existence of Nash equilibrium strategies in the class of stationary Markov strategies is proved for the ergodic-cost game. The key nontrivial contributions in the ergodic part are to prove the existence of a particular form of a (relative) value function solution to a player’s Bellman equation and the continuity of this solution with respect to the opponent’s strategies.

[1]  Ł. Stettner,et al.  Infinite horizon risk sensitive control of discrete time Markov processes with small risk , 2000 .

[2]  P. Whittle.  Risk-Sensitive Optimal Control , 1990 .

[3]  R. Bellman.  Dynamic programming. , 1957, Science.

[4]  Margriet B. Klompstra.  Nash equilibria in risk-sensitive dynamic games , 2000, IEEE Trans. Autom. Control..

[5]  Tomasz R. Bielecki,et al.  Economic Properties of the Risk Sensitive Criterion for Portfolio Management , 2003 .

[6]  A. Federgruen,et al.  A note on simultaneous recurrence conditions on a set of denumerable stochastic matrices : (preprint) , 1978 .

[7]  K. Fan.  Fixed-point and Minimax Theorems in Locally Convex Topological Linear Spaces. , 1952, Proceedings of the National Academy of Sciences of the United States of America.

[8]  T. Başar.  Nash Equilibria of Risk-Sensitive Nonlinear Stochastic Differential Games , 1999 .

[9]  Rommert Dekker,et al.  On the Relation Between Recurrence and Ergodicity Properties in Denumerable Markov Decision Chains , 1994, Math. Oper. Res..

[10]  O. Hernondex-lerma,et al.  Adaptive Markov Control Processes , 1989 .

[11]  Rommert Dekker,et al.  Recurrence Conditions for Average and Blackwell Optimality in Denumerable State Markov Decision Chains , 1992, Math. Oper. Res..

[12]  V. Borkar.  Topics in controlled Markov chains , 1991 .

[13]  Rolando Cavazos-Cadena,et al.  Controlled Markov chains with risk-sensitive criteria: Average cost, optimality equations, and optimal solutions , 1999, Math. Methods Oper. Res..

[14]  T. Sargent,et al.  Discounted linear exponential quadratic Gaussian control , 1995, IEEE Trans. Autom. Control..

[15]  Nicole Bäuerle,et al.  More Risk-Sensitive Markov Decision Processes , 2014, Math. Oper. Res..

[16]  Kim C. Border,et al.  Infinite Dimensional Analysis: A Hitchhiker’s Guide , 1994 .

[17]  Daniel Hernández-Hernández,et al.  Discounted Approximations for Risk-Sensitive Average Criteria in Markov Decision Chains with Finite State Space , 2011, Math. Oper. Res..

[18]  S. Balajiy,et al.  Multiplicative Ergodicity and Large Deviations for an Irreducible Markov Chain , 2000 .

[19]  R. Tweedie,et al.  Strengthening ergodicity to geometric ergodicity for markov chains , 1994 .

[20]  Hideo Nagai,et al.  Optimal Strategies for Risk-Sensitive Portfolio Optimization Problems for General Factor Models , 2002, SIAM J. Control. Optim..

[21]  Achim Klenke,et al.  Probability theory - a comprehensive course , 2008, Universitext.

[22]  S. Marcus,et al.  Risk sensitive control of Markov processes in countable state space , 1996 .

[23]  Uriel G. Rothblum,et al.  Multiplicative Markov Decision Chains , 1984, Math. Oper. Res..

[24]  Rhodes,et al.  Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games , 1973 .

[25]  S.,et al.  Risk-Sensitive Control and Dynamic Games for Partially Observed Discrete-Time Nonlinear Systems , 1994 .

[26]  V. Benes.  Existence of Optimal Strategies Based on Specified Information, for a Class of Stochastic Decision Problems , 1970 .

[27]  Mrinal K. Ghosh,et al.  Zero-Sum Risk-Sensitive Stochastic Differential Games , 2012, Math. Oper. Res..

[28]  P. Whittle.  Risk-sensitive linear/quadratic/gaussian control , 1981, Advances in Applied Probability.

[29]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[30]  Sean P. Meyn,et al.  Risk-Sensitive Optimal Control for Markov Decision Processes with Monotone Cost , 2002, Math. Oper. Res..

[31]  W. Fleming,et al.  Risk-Sensitive Control on an Infinite Time Horizon , 1995 .

[32]  Lukasz Stettner,et al.  Risk-Sensitive Control of Discrete-Time Markov Processes with Infinite Horizon , 1999, SIAM J. Control. Optim..

[33]  A. Nowak.  Notes on Risk-Sensitive Nash Equilibria , 2005 .

[34]  A. Hordijk,et al.  Contraction Conditions for Average and α-Discount Optimality in Countable State Markov Games with Unbounded Rewards , 1997, Math. Oper. Res..

[35]  Andrew E. B. Lim,et al.  Risk-sensitive control with HARA utility , 2001, IEEE Trans. Autom. Control..

[36]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[37]  Daniel Hernández-Hernández,et al.  Risk Sensitive Markov Decision Processes , 1997 .

[38]  Lukasz Stettner,et al.  Infinite Horizon Risk Sensitive Control of Discrete Time Markov Processes under Minorization Property , 2007, SIAM J. Control. Optim..

[39]  W. Fleming,et al.  Risk sensitive control of finite state machines on an infinite horizon. I , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[40]  Rolando Cavazos-Cadena,et al.  The vanishing discount approach in Markov chains with risk-sensitive criteria , 2000, IEEE Trans. Autom. Control..

[41]  V. Borkar.  Probability Theory: An Advanced Course , 1995 .

[42]  S. Hamadène,et al.  BSDEs and risk-sensitive control, zero-sum and nonzero-sum game problems of stochastic functional differential equations , 2003 .

[43]  M. K. Ghosh,et al.  Zero-sum risk-sensitive stochastic games on a countable state space , 2014 .