Discounted Markov Decision Problems