Theory of Optimal Stopping Problems

A very important subclass of the Markov Decision Problems considered so 3 far are optimal stopping problems. There, a Markov process (Xn)n∈N is given 4 which cannot be influenced by the decision maker. However, this process has 5 to be stopped at some time point n and a reward gn(Xn) is then obtained. 6 Thus, the only decision at each time point is whether the process should be 7 continued or stopped. Once it is stopped, no further decision is necessary. 8 Sometimes costs have to be paid or an additional reward is obtained as long 9 as the process is not stopped. Of course the aim is to find a stopping time 10 such that the expected stopping reward is maximized.