State space exploration in Markov models

Performance and dependability analysis is usually based on Markov models. One of the main problems faced by the analyst is the large state space cardinality of the Markov chain associated with the model, which precludes not only the model solution, but also the generation of the transition rate matrix. However, in many real system models, most of the probability mass is concentrated in a small number of states in comparison with the whole state space. Therefore, performability measures may be accurately evaluated from these “high probable” states. In this paper, we present an algorithm to generate the most probable states that is more efficient than previous algorithms in the literature. We also address the problem of calculating measures of interest and show how bounds on some measures can be efficiently calculated.

[1]  Pierre Semal,et al.  Bounds for the Positive Eigenvectors of Nonnegative Matrices and for their Approximations by Decomposition , 1984, JACM.

[2]  Kishor S. Trivedi.  Probability and Statistics with Reliability, Queuing, and Computer Science Applications , 1984 .

[3]  Edmundo de Souza e Silva,et al.  Performability Analysis of Computer Systems: From Model Spacification to Solution , 1992, Perform. Evaluation.

[4]  Daniel P. Heyman,et al.  Stochastic models in operations research , 1982 .

[5]  Jeffrey L. Arthur.  Stochastic Models in Operations Research, Volume I. (Daniel P. Heyman and Matthew J. Sobel) , 1983 .

[6]  Erhan Çinlar,et al.  Introduction to stochastic processes , 1974 .

[7]  Kishor S. Trivedi,et al.  Probabilistic modeling of computer system availability , 1987 .

[8]  Pierre Semal,et al.  Computable Bounds for Conditional Steady-State Probabilities in Large Markov Chains and Queueing Models , 1986, IEEE J. Sel. Areas Commun..

[9]  Mon-Song Chen,et al.  Dynamic State Exploration in Quantitative Protocol Analysis , 1989, PSTV.

[10]  John F. Meyer,et al.  On Evaluating the Performability of Degradable Computing Systems , 1980, IEEE Transactions on Computers.

[11]  Donald Gross,et al.  The Randomization Technique as a Modeling Tool and Solution Procedure for Transient Markov Processes , 1984, Oper. Res..

[12]  Miroslaw Malek,et al.  Survey of software tools for evaluating reliability, availability, and serviceability , 1988, CSUR.

[13]  Edmundo de Souza e Silva,et al.  Calculating availability and performability measures of repairable computer systems using randomization , 1989, JACM.

[14]  Louis A. Hageman,et al.  Iterative Solution of Large Linear Systems. , 1971 .

[15]  John F. Meyer,et al.  Performability: A Retrospective and Some Pointers to the Future , 1992, Perform. Evaluation.

[16]  Richard R. Muntz,et al.  Bounding Availability of Repairable Computer Systems , 1989, IEEE Trans. Computers.

[17]  Che-Liang Yang,et al.  Efficient computation of most probably states for communication networks with multimode components , 1989, IEEE Trans. Commun..

[18]  John A. Silvester,et al.  Performance Analysis of Networks with Unreliable Components , 1984, IEEE Trans. Commun..

[19]  S. Chiou,et al.  Reliability Analysis of a Communication Network with Multimode Components , 1986, IEEE J. Sel. Areas Commun..

[20]  de Souza e SilvaEdmundo,et al.  State space exploration in Markov models , 1992 .

[21]  Edmundo de Souza e Silva,et al.  Calculating Cumulative Operational Time Distributions of Repairable Computer Systems , 1986, IEEE Transactions on Computers.

[22]  Mon-Song Chen,et al.  An integrated algorithm for probabilistic protocol verification and evaluation , 1989, IEEE INFOCOM '89, Proceedings of the Eighth Annual Joint Conference of the IEEE Computer and Communications Societies.