Denumerable Markov Decision Chains: Sensitive Optimality Criteria

Assuming compact metric action spaces and the usual continuity properties of the immediate costs and of the transition probabilities we regard the existence of average and/or sensitive optimal stationary policies. We generalize results from the unichain case to the multichain case. It appears that the simultaneous Doeblin condition is not sufficient. However, the continuity of the ergodic potential guarantees not only average but also bias and Blackwell optimality. Relations between these conditions and uniform strong ergodicity are discussed. An extension is also made to the unbounded costs case.