Numerical evaluation of resolvents and Laplace transforms of Markov processes using linear programming

Abstract. This paper uses linear programming to numerically evaluate the Laplace transform of the exit time distribution and the resolvent of the moments of various Markov processes in bounded regions. The linear programming formulation is developed from a martingale characterization of the processes and the use of occupation measures. The LP approach naturally provides both upper and lower bounds on the quantities of interest. The processes analyzed include the Poisson process, one-dimensional Brownian motion (with and without drift), an Ornstein-Uhlenbeck process and two-dimensional Brownian motion. The Laplace transform of the original Cameron-Martin formula is also numerically evaluated by reducing it to the analysis of an Ornstein-Uhlenbeck process.

[1]  Richard H. Stockbridge,et al.  Long Term Average Control of a Local Time Process , 2002 .

[2]  W. T. Martin,et al.  Evaluation of various Wiener integrals by use of certain Sturm-Liouville differential equations , 1945 .

[3]  F. Knight.  Essentials of Brownian Motion and Diffusion , 1981 .

[4]  L. C. Young.  On approximation by polygons in the calculus of variations , 1933 .

[5]  V. Borkar,et al.  Occupation measures for controlled Markov processes: characterization and optimality , 1996 .

[6]  Kurt Helmes,et al.  Computing Moments of the Exit Time Distribution for Markov Processes by Linear Programming , 2001, Oper. Res..

[7]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[8]  H. Landau.  Moments in mathematics , 1987 .

[9]  Ioannis Karatzas,et al.  Brownian Motion and Stochastic Calculus , 1987 .

[10]  D. Widder,et al.  The Laplace Transform , 1943, The Mathematical Gazette.

[11]  R. Stockbridge.  Time-Average Control of Martingale Problems: Existence of a Stationary Solution , 1990 .

[12]  Thomas G. Kurtz,et al.  Martingale Problems for Constrained Markov Problems , 1990 .

[13]  R. Stockbridge.  Time-Average Control of Martingale Problems: A Linear Programming Formulation , 1990 .

[14]  J. Harrison,et al.  Steady-State Analysis of RBM in a Rectangle: Numerical Methods and A Queueing Application , 1991 .

[15]  Richard H. Stockbridge,et al.  Martingale problems and linear programs for singular control ∗ , 1999 .

[16]  J. Harrison,et al.  ON THE DISTRIBUTION OF MULTIDIMENSIONAL REFLECTED , 1981 .

[17]  J. Harrison,et al.  On the distribution of m ultidimen-sional reflected Brownian motion , 1981 .

[18]  T. Kurtz,et al.  Existence of Markov Controls and Characterization of Optimal Markov Controls , 1998 .