Discretisation of stochastic control problems for continuous time dynamics with delay

As a main step in the numerical solution of control problems in continuous time, the controlled process is approximated by sequences of controlled Markov chains, thus discretising time and space. A new feature in this context is to allow for delay in the dynamics. The existence of an optimal strategy with respect to the cost functional can be guaranteed in the class of relaxed controls. Weak convergence of the approximating extended Markov chains to the original process together with convergence of the associated optimal strategies is established.

[1]  H. Kushner.  Numerical approximations for nonlinear stochastic systems with delays , 2005 .

[2]  Xuerong Mao,et al.  Stochastic differential equations and their applications , 1997 .

[3]  N. Krylov.  On the rate of convergence of finite-difference approximations for Bellmans equations with variable coefficients , 2000 .

[4]  Bernt Øksendal,et al.  Some Solvable Stochastic Control Problems With Delay , 2000 .

[5]  P. Protter,et al.  Weak convergence of stochastic integrals and differential equations , 1996 .

[6]  Evelyn Buckwar,et al.  Introduction to the numerical analysis of stochastic delay differential equations , 2000 .

[7]  Eun-Kyung Lee,et al.  Projection Pursuit for Exploratory Supervised Classification , 2005 .

[8]  Salah-Eldin A. Mohammed,et al.  Discrete-time approximations of stochastic delay equations: The Milstein scheme , 2004 .

[9]  Raouf Boucekkine,et al.  Vintage capital and the dynamics of the AK model , 2005, J. Econ. Theory.

[10]  Patrick Florchinger,et al.  Convergence in Nonlinear Filtering for Stochastic Delay Systems , 2007, SIAM J. Control. Optim..

[11]  S. Mohammed.  Stochastic functional differential equations , 1984 .

[12]  J. Quadrat.  Numerical methods for stochastic control problems in continuous time , 1994 .

[13]  B. Øksendal,et al.  A maximum principle for optimal control of stochastic systems with delay, with applications to finance. , 2000 .

[14]  Alain Bensoussan,et al.  Representation and Control of Infinite Dimensional Systems (Systems & Control: Foundations & Applications) , 2006 .

[15]  B. Larssen.  Dynamic programming in stochastic control of systems with delay , 2002 .

[16]  H. Kushner.  Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems , 1990 .

[17]  P. Protter,et al.  Weak Limit Theorems for Stochastic Integrals and Stochastic Differential Equations , 1991 .

[18]  P. Protter.  Stochastic integration and differential equations , 1990 .

[19]  X. Zhou,et al.  Stochastic Controls: Hamiltonian Systems and HJB Equations , 1999 .

[20]  Xuerong Mao,et al.  Numerical solutions of stochastic functional differential equations , 2003 .