The Elimination algorithm for the problem of optimal stopping

Abstract. We present a new algorithm for solving the optimal stopping problem. The algorithm is based on the idea of elimination of states where stopping is nonoptimal and the corresponding correction of transition probabilities. The formal justification of this method is given by one of two presented theorems. The other theorem describes the situation when an aggregation of states is possible in the optimal stopping problem.

[1]  Theodore J. Sheskin,et al.  Technical Note - A Markov Chain Partitioning Algorithm for Computing Steady State Probabilities , 1985, Oper. Res..

[2]  John G. Kemeny,et al.  Finite Markov chains , 1960 .

[3]  F. Mosteller,et al.  Recognizing the Maximum of a Sequence , 1966 .

[4]  Mark H. A. Davis,et al.  A deterministic approach to optimal stopping with application to a prophet inequality , 1993 .

[5]  P. Freeman.  The Secretary Problem and its Extensions: A Review , 1983 .

[6]  Richard M. Feldman,et al.  Applied Probability and Stochastic Processes , 1995 .

[7]  Thomas S. Ferguson,et al.  Who Solved the Secretary Problem , 1989 .

[8]  Alʹbert Nikolaevich Shiri︠a︡ev,et al.  Optimal stopping rules , 1977 .

[9]  Ronald A. Howard,et al.  Dynamic Probabilistic Systems , 1971 .

[10]  Isaac M. Sonin,et al.  The Best Choice Problem for a Random Number of Objects , 1973 .

[11]  Erhan Çinlar,et al.  Introduction to stochastic processes , 1974 .

[12]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[13]  James O. Berger,et al.  The Relevance of Stopping Rules in Statistical Inference , 1988 .

[14]  David Siegmund,et al.  Great expectations: The theory of optimal stopping , 1971 .

[15]  G. Simons.  Great Expectations: Theory of Optimal Stopping , 1973 .

[16]  Robert J. Plemmons,et al.  Linear algebra, Markov chains, and queueing models , 1993 .

[17]  S. M. Samuels,et al.  Optimal selection based on relative rank (the “secretary problem”) , 1964 .

[18]  F Thomas Bruss,et al.  Strategies for Sequential Search and Selection in Real Time , 1992 .

[19]  David Lindley,et al.  Dynamic Programming and Decision Theory , 1961 .

[20]  John G. Kemeny,et al.  Finite Markov Chains. , 1960 .

[21]  Isaac M. Sonin,et al.  The State Reduction and Related Algorithms and Their Applications to the Study of Markov Chains, Graph Theory, and the Optimal Stopping Problem , 1999 .

[22]  Winfried K. Grassmann,et al.  Regenerative Analysis and Steady State Distributions for Markov Chains , 1985, Oper. Res..

[23]  Dimitri P. Bertsekas,et al.  Dynamic Programming: Deterministic and Stochastic Models , 1987 .