A characterization of exponential functionals in finite Markov chains

Abstract. This work considers Markov chains with finite state space. It is supposed that the process has a single recurrent class, but the set of transient states is not necessarily empty. In this context, a Varadhan’s function, giving the exponential grow rate of an aggregated cost function, is studied. The main result establishes that this functional is the optimal value of a minimization problem on the Euclidean space whose dimension equals the number of states.

[1]  O. Hernández-Lerma Adaptive Markov Control Processes , 1989 .

[2]  Ian R. Petersen,et al.  Robust Properties of Risk-Sensitive Control , 2000, Math. Control. Signals Syst..

[3]  Rolando Cavazos-Cadena,et al.  Controlled Markov chains with risk-sensitive criteria: Average cost, optimality equations, and optimal solutions , 1999, Math. Methods Oper. Res..

[4]  H. Nagai Bellman Equations of Risk-Sensitive Control , 1996 .

[5]  Hidehiro Kaise,et al.  Ergodic type Bellman equations of risk-sensitive control with large parameters and their singular limits , 1999 .

[6]  A. Bensoussan,et al.  Some Results on Risk-Sensitive Control with Full Observation , 1995 .

[7]  S. Meyn,et al.  Multiplicative ergodicity and large deviations for an irreducible Markov chain , 2000 .

[8]  J. Lynch,et al.  A weak convergence approach to the theory of large deviations , 1997 .

[9]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[10]  Shuenn-Jyi Sheu,et al.  On the Solutions of the Equation Arising from the Singular Limit of Some Eigen Problems , 1999 .

[11]  H. Nagai,et al.  Bellman–Isaacs equations of ergodic type related to risk‐sensitive control and their singular limits , 1998 .

[12]  S. Marcus,et al.  Risk sensitive control of Markov processes in countable state space , 1996 .

[13]  S. Sheu,et al.  ON THE STRUCTURE OF SOLUTIONS OF ERGODIC TYPE BELLMAN EQUATION RELATED TO RISK-SENSITIVE CONTROL , 2006, math/0602625.

[14]  R. Howard,et al.  Risk-Sensitive Markov Decision Processes , 1972 .