The Laurent series, sensitive discount and Blackwell optimality for continuous-time controlled Markov chains

Abstract.This paper gives conditions for the convergence of the Laurent series expansion for a class of continuous-time controlled Markov chains with possibly unbounded reward (or cost) rates and unbounded transition rates. That series is then used to study several optimization criteria, including n-discount optimality (for n=−1,0,1,...), Blackwell optimality, and the maximization of a certain vector criterion that in particular gives gain and bias optimality.

[1]  M. Puterman.  Sensitive Discount Optimality in Controlled One-Dimensional Diffusions , 1974 .

[2]  Arie Hordijk,et al.  Blackwell optimality in the class of all policies in Markov decision chains with a Borel state space and unbounded rewards , 1999, Math. Methods Oper. Res..

[3]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[4]  Howard M. Taylor,et al.  A laurent series for the resolvent of a strongly continuous stochastic semi-group , 1976 .

[5]  Arie Hordijk,et al.  Blackwell optimality in the class of stationary policies in Markov decision chains with a Borel state space and unbounded rewards , 1999, Math. Methods Oper. Res..

[6]  Onésimo Hernández-Lerma,et al.  Bias Optimality for Continuous-Time Controlled Markov Chains , 2006, SIAM J. Control. Optim..

[7]  B. L. Miller,et al.  Discrete Dynamic Programming with a Small Interest Rate , 1969 .

[8]  O. Hernández-Lerma,et al.  Continuous-time controlled Markov chains , 2003 .

[9]  D. Blackwell.  Discrete Dynamic Programming , 1962 .

[10]  A. Yushkevich.  On Reducing a Jump Controllable Markov Model to a Model with Discrete Time , 1980 .

[11]  Xianping Guo,et al.  Drift and monotonicity conditions for continuous-time controlled Markov chains with an average criterion , 2003, IEEE Trans. Autom. Control..

[12]  Xianping Guo,et al.  Continuous-Time Controlled Markov Chains with Discounted Rewards , 2003 .

[13]  A. F. Veinott.  Discrete Dynamic Programming with Sensitive Discount Optimality Criteria , 1969 .

[14]  O. Hernández-Lerma,et al.  Further topics on discrete-time Markov control processes , 1999 .

[15]  Onésimo Hernández-Lerma,et al.  Bias Optimality versus Strong 0-Discount Optimality in Markov Control Processes with Unbounded Costs , 2003 .

[16]  Tosio Kato.  Perturbation theory for linear operators , 1966 .