Computing Stationary Expectations in Level-Dependent QBD Processes

Stationary expectations corresponding to long-run averages of additive functionals on level-dependent quasi-birth-and-death processes are considered. Special cases include long-run average costs or rewards, moments and cumulants of steady-state queueing network performance measures, and many others. We provide a matrix-analytic scheme for numerically computing such stationary expectations without explicitly computing the stationary distribution of the process, which yields an algorithm that is as quick as its counterparts for stationary distributions but requires far less computer storage. Specific problems arising in the case of infinite state spaces are discussed and the application of the algorithm is demonstrated by a queueing network example.

[1]  T. Hanschke,et al.  A matrix continued fraction algorithm for the multiserver repeated order queue , 1999 .

[2]  R. Serfozo.  Basics of Applied Stochastic Processes , 2012 .

[3]  Peter G. Taylor,et al.  Calculating the equilibrium distribution in level dependent quasi-birth-and-death processes , 1995 .

[4]  Tom Burr,et al.  Introduction to Matrix Analytic Methods in Stochastic Modeling , 2001, Technometrics.

[5]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[6]  Hendrik Baumann,et al.  Steady state analysis of level dependent quasi-birth-and-death processes with catastrophes , 2012, Comput. Oper. Res..

[7]  Marcel F. Neuts,et al.  Matrix-Geometric Solutions in Stochastic Models , 1981 .

[8]  P. Jacobs,et al.  Finite birth-and-death models in randomly changing environments , 1984, Advances in Applied Probability.

[9]  M. Eisen,et al.  Probability and its applications , 1975 .

[10]  Upendra Dave,et al.  Applied Probability and Queues , 1987 .

[11]  P. Glynn,et al.  Bounding Stationary Expectations of Markov Processes , 2008 .

[12]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[13]  P. R. Kumar,et al.  Performance bounds for queueing networks and scheduling policies , 1994, IEEE Trans. Autom. Control..

[14]  Hendrik Baumann,et al.  Numerical solution of level dependent quasi-birth-and-death processes , 2010, ICCS.

[15]  Werner Sandmann,et al.  Infinite level-dependent QBD processes and matrix-analytic solutions for stochastic chemical kinetics , 2011, Advances in Applied Probability.

[16]  Panganamala Ramana Kumar,et al.  Computational Performance Bounds for Markov Chains With Applications , 2008, IEEE Transactions on Automatic Control.