Optimal Plans for Dynamic Programming Problems

It is proved that there exist stationary optimal plans for discounted dynamic programming problems, and that there exist semi-Markov e-optimal plans for positive dynamic programming problems. The actions are required to be taken in a variable action set Fs, and the reward function rs, a is a Borel measurable function of s, a and an u.s.c. function of a. Our results are related to recent work by Furukawa, Maitra, and Schal. The key tool is a generalization of a selection theorem of Dubins and Savage.