Probabilistic error analysis for some approximation schemes to optimal control problems

We introduce a class of numerical schemes for optimal control problems based on a novel Markov chain approximation, which uses, in turn, a piecewise constant policy approximation, Euler-Maruyama time stepping, and a Gauss-Hermite approximation of the Gaussian increments. We provide lower error bounds of order arbitrarily close to 1/2 in time and 1/3 in space for Lipschitz viscosity solutions, coupling probabilistic arguments with regularization techniques as introduced by Krylov. The corresponding order of the upper bounds is 1/4 in time and 1/5 in space. For sufficiently regular solutions, the order is 1 in both time and space for both bounds. Finally, we propose techniques for further improving the accuracy of the individual components of the approximation.

[1]  Maria Tchernychova.  Carathéodory cubature measures , 2015 .

[2]  Guy Barles,et al.  Error bounds for monotone approximation schemes for parabolic Hamilton-Jacobi-Bellman equations , 2007, Math. Comput..

[3]  T. Faniran.  Numerical Solution of Stochastic Differential Equations , 2015 .

[4]  Yaacov Z. Bergman.  Option Pricing with Differential Interest Rates , 1995 .

[5]  N. Touzi.  Optimal Stochastic Control, Stochastic Target Problems, and Backward SDE , 2012 .

[6]  G. Milstein.  Numerical Integration of Stochastic Differential Equations , 1994 .

[7]  Terry Lyons,et al.  Cubature on Wiener space , 2004, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[8]  E Weinan,et al.  Deep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential Equations , 2017, Communications in Mathematics and Statistics.

[9]  N. Krylov.  On the rate of convergence of finite-difference approximations for Bellmans equations with variable coefficients , 2000 .

[10]  P. Lions,et al.  User’s guide to viscosity solutions of second order partial differential equations , 1992, math/9207212.

[11]  Christian Bender,et al.  A PRIMAL–DUAL ALGORITHM FOR BSDES , 2013, 1310.3694.

[12]  G. Barles,et al.  Convergence of approximation schemes for fully nonlinear second order equations , 1990, 29th IEEE Conference on Decision and Control.

[13]  Robert Denk,et al.  A forward scheme for backward SDEs , 2007 .

[14]  Walter Murray Wonham,et al.  Optimal stochastic control , 1969, Autom..

[15]  Harold J. Kushner,et al.  Optimal stochastic control , 1962 .

[16]  Jose-Luis Mendali.  Some estimates for finite difference approximations , 1989 .

[17]  H. Kushner.  Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[18]  Paul V. Preckel,et al.  Gaussian cubature: A practitioner's guide , 2007, Math. Comput. Model..

[19]  Arnold Neumaier,et al.  Introduction to Numerical Analysis , 2001 .

[20]  Maurizio Falcone,et al.  An approximation scheme for the optimal control of diffusion processes , 1995 .

[21]  Christoph Reisinger,et al.  Duality-based a posteriori error estimates for some approximation schemes for optimal investment problems , 2020, Comput. Math. Appl..

[22]  E. Gobet,et al.  A regression-based Monte Carlo method to solve backward stochastic differential equations , 2005, math/0508491.

[23]  Kristian Debrabant,et al.  Semi-Lagrangian schemes for linear and fully non-linear diffusion equations , 2009, Math. Comput..

[24]  George B. Dantzig,et al.  A PRIMAL--DUAL ALGORITHM , 1956 .

[25]  Guy Barles,et al.  On the convergence rate of approximation schemes for Hamilton-Jacobi-Bellman equations , 2002 .

[26]  N. Krylov,et al.  Approximating Value Functions for Controlled Degenerate Diffusion Processes by Using Piece-Wise Constant Policies , 1999 .

[27]  P. Forsyth,et al.  Numerical methods for controlled Hamilton-Jacobi-Bellman PDEs in finance , 2007 .

[28]  X. Zhou,et al.  Stochastic Controls: Hamiltonian Systems and HJB Equations , 1999 .

[29]  Josef Teichmann,et al.  The proof of Tchakaloff’s Theorem , 2005 .

[30]  Anna Lisa Amadori,et al.  Nonlinear integro-differential evolution problems arising in option pricing: a viscosity solutions approach , 2003, Differential and Integral Equations.

[31]  Christoph Reisinger,et al.  A Penalty Method for the Numerical Solution of Hamilton-Jacobi-Bellman (HJB) Equations in Finance , 2011, SIAM J. Numer. Anal..

[32]  William H. Beyer,et al.  CRC standard mathematical tables , 1976 .

[33]  Jean-François Richard,et al.  Methods of Numerical Integration , 2000 .

[34]  Guy Barles,et al.  Error Bounds for Monotone Approximation Schemes for Hamilton-Jacobi-Bellman Equations , 2005, SIAM J. Numer. Anal..

[35]  Christoph Reisinger,et al.  Piecewise constant policy approximations to Hamilton-Jacobi-Bellman equations , 2015, 1503.05864.

[36]  Thomas Gerstner,et al.  Numerical integration using sparse grids , 2004, Numerical Algorithms.

[37]  Kristian Debrabant,et al.  Semi-Lagrangian schemes for parabolic equations , 2013 .

[38]  R. E. Carlson,et al.  Monotone Piecewise Cubic Interpolation , 1980 .

[39]  Fabio Mercurio,et al.  Bergman, Piterbarg and Beyond: Pricing Derivatives under Collateralization and Differential Rates , 2013 .