Flexible forward improvement iteration for infinite time horizon Markovian optimal stopping problems

In this paper, we propose an extension of the forward improvement iteration algorithm, originally introduced in Irle [2006] and recently reconsidered in Miclo and Villeneuve [2021]. The main new ingredient is a flexible window parameter describing the look-ahead distance in the improvement step. We consider the framework of a Markovian optimal stopping problem in discrete time with random discounting and infinite time horizon. We prove convergence and show that the additional flexibility may significantly reduce the runtime.

[1]  Sören Christensen,et al.  On Finding Equilibrium Stopping Times for Time-Inconsistent Markovian Problems , 2017, SIAM J. Control. Optim..

[2]  John N. Tsitsiklis,et al.  Optimal stopping of Markov processes: Hilbert space theory, approximation algorithms, and an application to pricing high-dimensional financial derivatives , 1999, IEEE Trans. Autom. Control..

[3]  Isaac Sonin,et al.  The Elimination algorithm for the problem of optimal stopping , 1999, Math. Methods Oper. Res..

[4]  S. Christensen.  A METHOD FOR PRICING AMERICAN OPTIONS USING SEMI‐INFINITE LINEAR PROGRAMMING , 2011, 1103.4483.

[5]  John Schoenmakers,et al.  Iterative construction of the optimal Bermudan stopping time , 2006, Finance Stochastics.

[6]  G. Simons.  Great Expectations: Theory of Optimal Stopping , 1973 .

[7]  Stéphane Villeneuve,et al.  On the forward algorithm for stopping problems on continuous-time Markov chains , 2021, Journal of Applied Probability.

[8]  Richard H. Stockbridge,et al.  Linear Programming Formulation for Optimal Stopping Problems , 2001, SIAM J. Control. Optim..

[9]  E. Presman.  A new approach to the solution of optimal stopping problem in a discrete time , 2011 .

[10]  Albert N. Shiryaev,et al.  Optimal Stopping Rules , 2011, International Encyclopedia of Statistical Science.

[11]  Upendra Dave,et al.  Applied Probability and Queues , 1987 .

[12]  Albrecht Irle,et al.  On the best choice problem with random population size , 1980, Z. Oper. Research.

[13]  Albrecht Irle,et al.  The monotone case approach for the solution of certain multidimensional optimal stopping problems , 2017 .

[14]  A. Cox.  Optimal Stopping and Applications , 2009 .

[15]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[16]  Tobias Sohr,et al.  A solution technique for Lévy driven long term average impulse control problems , 2019 .

[17]  A. Irle.  A FORWARD ALGORITHM FOR SOLVING OPTIMAL STOPPING PROBLEMS , 2006 .

[18]  J. Schoenmakers,et al.  An iterative method for multiple stopping: convergence and stability , 2006, Advances in Applied Probability.