Immediate Events in Markov Chains

Immediate events are events which happen without delay. There are several contexts in which immediate events occur. In particular, they are important for formulating Markovian systems efficiently, they are an integral part of generalised stochastic Petri nets, and they are useful for analysing processes which have different time scales. These applications are reviewed, and a unified theory for finding equilibrium probabilities for such processes is developed. This theory is based on the GTH algorithm, which is augmented by a special algebra.

[1]  Victor L. Wallace,et al.  RQA-1 : the recursive queue analyser , 1966 .

[2]  C. O'Cinneide.  Entrywise perturbation theory and error analysis for Markov chains , 1993 .

[3]  S. Sastry,et al.  Hierarchical aggregation of singularly perturbed finite state Markov processes , 1983 .

[4]  Alan S. Willsky,et al.  The reduction of perturbed Markov generators: an algorithm exposing the role of transient states , 1988, JACM.

[5]  Winfried K. Grassmann,et al.  Equilibrium distribution of block-structured Markov chains with repeating rows , 1990, Journal of Applied Probability.

[6]  G. Stewart,et al.  On a direct method for the solution of nearly uncoupled Markov chains , 1991 .

[7]  Winfried K. Grassmann,et al.  Regenerative Analysis and Steady State Distributions for Markov Chains , 1985, Oper. Res..

[8]  Marco Ajmone Marsan,et al.  A class of generalized stochastic Petri nets for the performance evaluation of multiprocessor systems , 1984, TOCS.

[9]  W. K. Grassman,et al.  The Factorization of Queueing Equations and Their Interpretation , 1985 .

[10]  Donald Gross,et al.  The Randomization Technique as a Modeling Tool and Solution Procedure for Transient Markov Processes , 1984, Oper. Res..

[11]  Kishor S. Trivedi,et al.  On the Solution of GSPN Reward Models , 1991, Perform. Evaluation.

[12]  Winfried K. Grassmann,et al.  Rounding errors in certain algorithms involving Markov chains , 1993, TOMS.

[13]  Dianne P. O'Leary,et al.  A Block-GTH Algorithm for Finding the Stationary Vector of a Markov Chain , 1996, SIAM J. Matrix Anal. Appl..