Existence of Risk-Sensitive Optimal Stationary Policies for Controlled Markov Processes

Abstract. In this paper we are concerned with the existence of optimal stationary policies for infinite-horizon risk-sensitive Markov control processes with denumerable state space, unbounded cost function, and long-run average cost. Introducing a discounted cost dynamic game, we prove that its value function satisfies an Isaacs equation, and its relationship with the risk-sensitive control problem is studied. Using the vanishing discount approach, we prove that the risk-sensitive dynamic programming inequality holds, and derive an optimal stationary policy.

[1]  V. Borkar On Minimum Cost Per Unit Time Control of Markov Chains , 1984 .

[2]  L. Sennott A new condition for the existence of optimal stationary policies in average cost Markov decision processes , 1986 .

[3]  Rolando Cavazos-Cadena Weak conditions for the existence of optimal stationary policies in average Markov decision chains with unbounded costs , 1989, Kybernetika.

[4]  P. Whittle Risk-Sensitive Optimal Control , 1990 .

[5]  O. Hernández-Lerma,et al.  Average cost optimal policies for Markov control processes with Borel state space and unbounded costs , 1990 .

[6]  J. Filar,et al.  Some comments on a theorem of Hardy and Littlewood , 1992 .

[7]  Rolando Cavazos-Cadena,et al.  Comparing recent assumptions for the existence of average optimal stationary policies , 1992, Oper. Res. Lett..

[8]  M. K. Ghosh,et al.  Discrete-time controlled Markov processes with average cost criterion: a survey , 1993 .

[9]  Onésimo Hernández-Lerma,et al.  Weak conditions for average optimality in Markov control processes , 1994 .

[10]  S.I. Marcus,et al.  Risk-sensitive optimal control of hidden Markov models: a case study , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[11]  M. James,et al.  Robust and Risk-Sensitive Output Feedback Control for Finite State Machines and Hidden Markov Models , 1994 .

[12]  S. Marcus,et al.  Risk sensitive control of Markov processes in countable state space , 1996 .

[13]  W. Fleming,et al.  Risk-Sensitive Control of Finite State Machines on an Infinite Horizon I , 1997 .

[14]  J. Lynch,et al.  A weak convergence approach to the theory of large deviations , 1997 .

[15]  W. Fleming Book Review: Discrete-time Markov control processes: Basic optimality criteria , 1997 .

[16]  O. Hernández-Lerma,et al.  Discrete-time Markov control processes , 1999 .