Regenerative Markov decision models

Discrete time Markov decision processes with a countable state space are investigated. Under a condition guaranteeing the recurrence to a fixed state, the existence of stationary optimal policies with respect to discounted expected and average expected return is shown. Also sensitive discount optimal policies do exist and limit decision rules, as the discount-factor tends to one, of discounted optimal rules are bias or equivalently average-overtaken optimal. Finally, an iteration procedure to compute sensitive discount optimal policies is given.