Optimal Stochastic Switching under Convexity Assumptions

We address a method of approximate calculation of optimal control policy applicable to a particular class of stochastic control problems, whose stochastic dynamics exhibit a certain convexity preserving property. Problems of this type appear in many applications and encompass important examples arising in the area of optimal stopping and in the framework of control, based on partial observations. Utilizing this specific structure, we suggest a numerical method which enjoys a number of desirable properties. In particular, we work out a remarkably strong method for calculation of the value function: Within our numerically well-trackable approach, we show a convergence to the value function of the original problem uniformly on compact sets. This issue can be of great advantage, particularly for high-dimensional control problem, where the only competing methods from least-squares Monte Carlo family are able to serve merely $L^{p}$-convergence, under several restrictions. Since the presented algorithm is simpl...

[1]  Jianqing Fan,et al.  Local polynomial modelling and its applications , 1994 .

[2]  David Williams,et al.  Probability with Martingales , 1991, Cambridge mathematical textbooks.

[3]  M. Kohler,et al.  A DYNAMIC LOOK-AHEAD MONTE CARLO ALGORITHM FOR PRICING AMERICAN OPTIONS , 2006 .

[4]  John N. Tsitsiklis,et al.  Regression methods for pricing complex American-style options , 2001, IEEE Trans. Neural Networks.

[5]  Onésimo Hernández Lerma,et al.  Monotone approximations for convex stochastic control problems , 1992 .

[6]  Denis Belomestny,et al.  Regression Methods for Stochastic Control Problems and Their Convergence Analysis , 2009, SIAM J. Control. Optim..

[7]  Peter W. Glynn,et al.  Kernel-Based Reinforcement Learning in Average-Cost Problems: An Application to Optimal Portfolio Choice , 2000, NIPS.

[8]  J. Carriére.  Valuation of the early-exercise price for options using simulations and nonparametric regression , 1996 .

[9]  Lars Stentoft,et al.  Convergence of the Least Squares Monte Carlo Approach to American Option Valuation , 2004, Manag. Sci..

[10]  R. Carmona,et al.  OPTIMAL MULTIPLE STOPPING AND VALUATION OF SWING OPTIONS , 2008 .

[11]  Paul Glasserman,et al.  Monte Carlo Methods in Financial Engineering , 2003 .

[12]  Philip Protter,et al.  An analysis of the Longstaff-Schwartz algorithm for American option pricing , 2001 .

[13]  John N. Tsitsiklis,et al.  Optimal stopping of Markov processes: Hilbert space theory, approximation algorithms, and an application to pricing high-dimensional financial derivatives , 1999, IEEE Trans. Autom. Control..

[14]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[15]  Daniel Egloff.  Monte Carlo algorithms for optimal stopping and statistical learning , 2004, math/0408276.

[16]  Francis A. Longstaff,et al.  Valuing American Options by Simulation: A Simple Least-Squares Approach , 2001 .

[17]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[18]  U. Rieder,et al.  Markov Decision Processes with Applications to Finance , 2011 .

[19]  Warren B. Powell,et al.  Approximate Dynamic Programming - Solving the Curses of Dimensionality , 2007 .

[20]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[21]  Liming Xiang,et al.  Kernel-Based Reinforcement Learning , 2006, ICIC.

[22]  Milos Hauskrecht,et al.  Value-Function Approximations for Partially Observable Markov Decision Processes , 2000, J. Artif. Intell. Res..