In many special models the existence of maximizers at all stages (and hence by the OC in Theorem 2.3.3(c) also of s-optimal action sequences for all initial states s) can be established by ad hoc methods. The existence of a maximizer at each stage is also obvious if the set D(s) of admissible actions is finite for all s. In Proposition 7.1.10 we gave a result which covers many applications where S and A are intervals. The existence problem for maximizers under more general conditions is most easily dealt with under assumptions which are so strong that continuity (or at least upper semicontinuity) of W n and also of V n is implied.
 David Lindley,et al. How to Gamble If You Must (Inequalities for Stochastic Processes) , 1966 .