A Symbolic Out-of-Core Solution Method for Markov Models

Despite considerable eort, the state-space explosion problem remains an issue in the analysis of Markov models. Given structure, symbolic representations can result in very compact encoding of the models. However, a major obstacle for symbolic methods is the need to store the probability vector(s) explicitly in main memory. In this paper, we present a novel algorithm which relaxes these memory limitations by storing the probability vector on disk. The algorithm has been implemented using an MTBDD-based data structure to store the matrix and an array to store the vector. We report on experimental results for two benchmark models, a Kanban manufacturing system and a flexible manufacturing system, with models as large as 133 million states. Discrete-state Markovian models are widely employed for the analysis of communication networks and computer systems. It is often convenient to model such systems as Continuous Time Markov Chains (CTMCs), provided probability distributions are assumed to be exponential. A CTMC may be represented by a set of states and a transition rate matrix Q containing state transition rates as coecients. A CTMC can be analysed using probabilistic model checking. Required or desired performance properties are specified as formulas in the temporal logic CSL and then automatically verified using the appropriate model checking algorithms. A core component of these algorithms is the computation of the steady-state probabilities of the CTMC. This is reducible to the classical problem of solving a system of linear equations of the form Ax = b where b = 0. A range of solution techniques exist to combat the

[1]  Sivan Toledo,et al.  A survey of out-of-core algorithms in numerical linear algebra , 1999, External Memory Algorithms.

[2]  Marta Z. Kwiatkowska,et al.  PRISM: Probabilistic Symbolic Model Checker , 2002, Computer Performance Evaluation / TOOLS.

[3]  Masahiro Fujita,et al.  Multi-Terminal Binary Decision Diagrams: An Efficient Data Structure for Matrix Representation , 1997, Formal Methods Syst. Des..

[4]  Holger Hermanns,et al.  On the use of MTBDDs for performability analysis and verification of stochastic systems , 2003, J. Log. Algebraic Methods Program..

[5]  William H. Sanders,et al.  An Efficient Disk-Based Tool for Solving Large Markov Models , 1998, Perform. Evaluation.

[6]  David Anthony Parker,et al.  Implementation of symbolic model checking for probabilistic systems , 2003 .

[7]  William H. Sanders,et al.  From Performance Evaluation 1998. An ecient disk-based tool for solving large Markov models , 1998 .

[8]  William H. Sanders,et al.  An Efficient Disk-Based Tool for Solving Very Large Markov Models , 1997, Computer Performance Evaluation.

[9]  Gianfranco Ciardo.  What a Structural World , 2001 .

[10]  M. Siegle,et al.  Multi Terminal Binary Decision Diagrams to Represent and Analyse Continuous Time Markov Chains , 1999 .

[11]  R. I. Bahar,et al.  Algebraic decision diagrams and their applications , 1993, Proceedings of 1993 International Conference on Computer Aided Design (ICCAD).

[12]  G. Ciardo,et al.  ON THE USE OF KRONECKER OPERATORS FOR THE SOLUTION OF GENERALIZED STOCHASTIC PETRI NETS , 1996 .

[13]  Kishor S. Trivedi,et al.  A Decomposition Approach for Stochastic Reward Net Models , 1993, Perform. Evaluation.

[14]  William H. Sanders,et al.  "On-the-Fly'' Solution Techniques for Stochastic Petri Nets and Extensions , 1998, IEEE Trans. Software Eng..

[15]  Peter Buchholz,et al.  Compact representations of probability distributions in the analysis of Superposed GSPNs , 2001, Proceedings 9th International Workshop on Petri Nets and Performance Models.

[16]  Markus Siegle,et al.  Advances in Model Representations , 2001, PAPM-PROBMIV.

[17]  S TrivediKishor,et al.  A decomposition approach for stochastic reward net models , 1993 .

[18]  Brigitte Plateau.  On the stochastic structure of parallelism and synchronization models for distributed algorithms , 1985, SIGMETRICS 1985.

[19]  I. Duff,et al.  Direct Methods for Sparse Matrices , 1987 .

[20]  Marta Z. Kwiatkowska,et al.  Probabilistic symbolic model checking with PRISM: a hybrid approach , 2004, International Journal on Software Tools for Technology Transfer.

[21]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[22]  Enrico Macii,et al.  Markovian analysis of large finite state machines , 1996, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[23]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[24]  Andrew S. Miner.  Efficient solution of GSPNs using canonical matrix diagrams , 2001, Proceedings 9th International Workshop on Petri Nets and Performance Models.

[25]  Marta Z. Kwiatkowska,et al.  Out-of-Core Solution of Large Linear Systems of Equations Arising from Stochastic Modelling , 2002, PAPM-PROBMIV.

[26]  Gianfranco Ciardo,et al.  A data structure for the efficient Kronecker solution of GSPNs , 1999, Proceedings 8th International Workshop on Petri Nets and Performance Models (Cat. No.PR00331).