Transient Solutions of Markov Processes by Krylov Subspaces

In this note we exploit the knowledge embodied in infinitesimal generators of Markov processes to compute efficiently and economically the transient solution of continuous time Markov processes. We consider the Krylov subspace approximation method which has been analysed by Gallopoulos and Saad for solving partial differential equations and linear ordinary differential equations [7, 17]. We place special emphasis on error bounds and stepsize control. We illustrate the usefulness of the approach by providing some application examples.

[1]  S. Agmon,et al.  Estimates near the boundary for solutions of elliptic partial differential equations satisfying general boundary conditions. I , 1959 .

[2]  R. Varga.  Nonnegatively posed problems and completely monotonic functions , 1968 .

[3]  R. Varga,et al.  Chebyshev rational approximations to e−x in [0, +∞) and applications to heat-conduction problems , 1969 .

[4]  E. Saff.  On the degree of best rational approximation to the exponential function , 1973 .

[5]  B. Parlett.  A recurrence among the elements of functions of triangular matrices , 1976 .

[6]  C. Loan.  The Sensitivity of the Matrix Exponential , 1977 .

[7]  R. Ward.  Numerical Computation of the Matrix Exponential with Accuracy Estimate , 1977 .

[8]  C. Loan,et al.  Nineteen Dubious Ways to Compute the Exponential of a Matrix , 1978 .

[9]  Marcel F. Neuts,et al.  Matrix-geometric solutions in stochastic models - an algorithmic approach , 1982 .

[10]  Valerie Isham,et al.  Non‐Negative Matrices and Markov Chains , 1983 .

[11]  Gene H. Golub,et al.  Matrix computations , 1983 .

[12]  R. Varga,et al.  Extended numerical computations on the “1/9” conjecture in rational approximation theory , 1984 .

[13]  B N Parlett,et al.  Development of an Accurate Algorithm for EXP(Bt). Appendix , 1985 .

[14]  Beresford N. Parlett,et al.  Programs to Swap Diagonal Blocks , 1987 .

[15]  Kishor S. Trivedi,et al.  Transient analysis of cumulative measures of markov model behavior , 1989 .

[16]  Kjell Gustafsson,et al.  Control theoretic techniques for stepsize selection in explicit Runge-Kutta methods , 1991, TOMS.

[17]  Yousef Saad,et al.  Efficient Solution of Parabolic Equations by Krylov Approximation Methods , 1992, SIAM J. Sci. Comput..

[18]  Y. Saad.  Analysis of some Krylov subspace approximations to the matrix exponential operator , 1992 .

[19]  J K Muppala,et al.  Higher-order Methods for Transient Analysis of Stii Markov Chains. in Surf: a Program for Dependability Evaluation of Complex Fault-tolerant Computing Systems. in Proc. 11th Intl. Symposium on Fault-tolerant Computing, Automated Generation and Analysis of Markov Reward Models Using Stochastic Reward , 1993 .

[20]  Computing selected eigenvalues of sparse unsymmetric matrices using subspace iteration , 1993, TOMS.

[21]  Raymond A. Marie,et al.  The uniformized power method for transient solutions of Markov processes , 1993, Comput. Oper. Res..

[22]  Robert J. Plemmons,et al.  Nonnegative Matrices in the Mathematical Sciences , 1979, Classics in Applied Mathematics.

[23]  Bernard Philippe,et al.  Numerical Methods in Markov Chain Modeling , 1992, Oper. Res..