Solution of Large Markov Models Using Lumping Techniques and Symbolic Data Structures

Continuous time Markov chains (CTMCs) are among the most fundamental mathematical structures used for performance and dependability modeling of communication and computer systems. They are often constructed from models described in one of the various high-level formalisms. Since the size of a CTMC usually grows exponentially with the size of the corresponding high-level model, one often encounters the infamous state-space explosion problem, which often makes solution of the CTMCs intractable. In state-based numerical analysis, the solution technique we have chosen to use to solve for measures defined on a CTMC, the state-space explosion problem is manifested in two ways: (1) large state transition rate matrices, and (2) large iteration vectors. The goal of this dissertation is to extend, improve, and combine existing solutions of the state-space explosion problem in order to make possible the construction and solution of very large CTMCs generated from high-level models. Our new techniques follow largeness avoidance and largeness tolerance approaches. In the former approach, we reduce the size of the CTMC that needs to be solved in order to compute the desired measures. That makes both the transition matrix and the iteration vectors smaller. In the latter approach, we reduce the size of the representation of the transition matrix by using symbolic data structures. In particular, we have developed the fastest known CTMC lumping algorithm with the running time of O (m log n), where n and m are the number of states and non-zero entries of the generator matrix of the CTMC, respectively. We have also combined the use of symbolic data structures with state-lumping techniques to develop an efficient symbolic state-space exploration algorithm for state-sharing composed models that exploits lumpings that are due to equally behaving components. Finally, we have developed a new compositional algorithm that lumps CTMCs represented as MDs. Unlike other compositional lumping algorithms, our algorithm does not require any knowledge of the modeling formalisms from which the MDs were generated. Our approach relies on local conditions, i.e., conditions on individual nodes of the MD.

[1]  Gianfranco Ciardo,et al.  Data structures for the analysis of large structured markov models , 2000 .

[2]  William H. Sanders,et al.  From Performance Evaluation 1998. An ecient disk-based tool for solving large Markov models , 1998 .

[3]  Toshihide Ibaraki,et al.  On-Line Computation of Transitive Closures of Graphs , 1983, Inf. Process. Lett..

[4]  Robert K. Brayton,et al.  Algorithms for discrete function manipulation , 1990, 1990 IEEE International Conference on Computer-Aided Design. Digest of Technical Papers.

[5]  David Parker,et al.  Symbolic Representations and Analysis of Large Probabilistic Systems , 2004, Validation of Stochastic Systems.

[6]  Y. Gardan,et al.  Performance evaluation with asynchronously decomposable SWN: implementation and case study , 2003, 10th International Workshop on Petri Nets and Performance Models, 2003. Proceedings..

[7]  Scott A. Smolka,et al.  CCS expressions, finite state processes, and three problems of equivalence , 1983, PODC '83.

[8]  William H. Sanders,et al.  "On-the-Fly'' Solution Techniques for Stochastic Petri Nets and Extensions , 1998, IEEE Trans. Software Eng..

[9]  P. Buchholz.  Equivalence Relations for Stochastic Automata Networks , 1995 .

[10]  Joost-Pieter Katoen,et al.  MoDeST - A Modelling and Description Language for Stochastic Timed Systems , 2001, PAPM-PROBMIV.

[11]  William H. Sanders,et al.  Stochastic Activity Networks: Formal Definitions and Concepts , 2002, European Educational Forum: School on Formal Methods and Performance Analysis.

[12]  Marco Ajmone Marsan,et al.  A class of generalized stochastic Petri nets for the performance evaluation of multiprocessor systems , 1984, TOCS.

[13]  Edmund M. Clarke,et al.  Symbolic Model Checking: 10^20 States and Beyond , 1990, Inf. Comput..

[14]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[15]  Yung-Te Lai,et al.  Edge-valued binary decision diagrams for multi-level hierarchical verification , 1992, DAC '92.

[16]  Edmund M. Clarke,et al.  Model Checking , 1999, Handbook of Automated Reasoning.

[17]  William H. Sanders,et al.  Stochastic Activity Networks: Structure, Behavior, and Application , 1985, PNPM.

[18]  Holger Hermanns,et al.  Exploiting Symmetries in Stochastic Process Algebras , 1998, ESM.

[19]  William H. Sanders,et al.  An Approach for Bounding Reward Measures in Markov Models Using Aggregation , 2004 .

[20]  William H. Sanders,et al.  Reduced Base Model Construction Methods for Stochastic Activity Networks , 1991, IEEE J. Sel. Areas Commun..

[21]  Robin Milner,et al.  Communication and concurrency , 1989, PHI Series in computer science.

[22]  Andrew S. Miner.  Efficient solution of GSPNs using canonical matrix diagrams , 2001, Proceedings 9th International Workshop on Petri Nets and Performance Models.

[23]  Brigitte Plateau.  On the stochastic structure of parallelism and synchronization models for distributed algorithms , 1985, SIGMETRICS 1985.

[24]  Marco Ajmone Marsan,et al.  Modelling with Generalized Stochastic Petri Nets , 1995, PERV.

[25]  William H. Sanders,et al.  Optimal state-space lumping in Markov chains , 2003, Inf. Process. Lett..

[26]  Jean-Claude Fernandez,et al.  An Implementation of an Efficient Algorithm for Bisimulation Equivalence , 1990, Sci. Comput. Program..

[27]  John G. Kemeny,et al.  Finite Markov chains , 1960 .

[28]  Yao Li,et al.  Performance Petri net analysis of communications protocol software by delay-equivalent aggregation , 1991, Proceedings of the Fourth International Workshop on Petri Nets and Performance Models PNPM91.

[29]  Andrew S. Miner,et al.  Computing response time distributions using stochastic Petri nets and matrix diagrams , 2003, 10th International Workshop on Petri Nets and Performance Models, 2003. Proceedings..

[30]  Robert E. Tarjan,et al.  Three Partition Refinement Algorithms , 1987, SIAM J. Comput..

[31]  Robert K. Brayton,et al.  Verifying Continuous Time Markov Chains , 1996, CAV.

[32]  Jane Hillston,et al.  A compositional approach to performance modelling , 1996 .

[33]  William H. Sanders,et al.  Construction and solution of performability models based on stochastic activity networks , 1988 .

[34]  William H. Sanders,et al.  A Structured path-based approach for computing transient rewards of large CTMCs , 2004 .

[35]  Anne Benoit,et al.  Aggregation of stochastic automata networks with replicas , 2004 .

[36]  Peter Buchholz,et al.  Complexity of Memory-Efficient Kronecker Operations with Applications to the Solution of Markov Models , 2000, INFORMS J. Comput..

[37]  Brigitte Plateau,et al.  Stochastic Automata Network For Modeling Parallel Systems , 1991, IEEE Trans. Software Eng..

[38]  Giovanni Chiola,et al.  Stochastic Well-Formed Colored Nets and Symmetric Modeling Applications , 1993, IEEE Trans. Computers.

[39]  Chris M. N. Tofts,et al.  A Synchronous Calculus of Relative Frequency , 1990, CONCUR.

[40]  Holger Hermanns,et al.  Bisimulation Algorithms for Stochastic Process Algebras and Their BDD-Based Implementation , 1999, ARTS.

[41]  Edmundo de Souza e Silva,et al.  Calculating availability and performability measures of repairable computer systems using randomization , 1989, JACM.

[42]  Randal E. Bryant,et al.  Graph-Based Algorithms for Boolean Function Manipulation , 1986, IEEE Transactions on Computers.

[43]  Christel Baier,et al.  On the Logical Characterisation of Performability Properties , 2000, ICALP.

[44]  Holger Hermanns,et al.  Interactive Markov Chains , 2002, Lecture Notes in Computer Science.

[45]  Roberto Gorrieri,et al.  A Tutorial on EMPA: A Theory of Concurrent Processes with Nondeterminism, Priorities, Probabilities and Time , 1998, Theor. Comput. Sci..

[46]  Kim G. Larsen,et al.  Bisimulation through Probabilistic Testing , 1991, Inf. Comput..

[47]  Gianfranco Ciardo,et al.  A data structure for the efficient Kronecker solution of GSPNs , 1999, Proceedings 8th International Workshop on Petri Nets and Performance Models (Cat. No.PR00331).

[48]  Jean-Michel Fourneau,et al.  Lumpable continuous-time stochastic automata networks , 2003, Eur. J. Oper. Res..

[49]  Chris M. N. Tofts,et al.  Processes with probabilities, priority and time , 1994, Formal Aspects of Computing.

[50]  Pierre Semal,et al.  Computable Bounds for Conditional Steady-State Probabilities in Large Markov Chains and Queueing Models , 1986, IEEE J. Sel. Areas Commun..

[51]  W. D. Obal,et al.  Measure-adaptive state-space construction methods , 1998 .

[52]  William H. Sanders,et al.  A Unified Approach for Specifying Measures of Performance, Dependability and Performability , 1991 .

[53]  Muhammad Akber Qureshi,et al.  Construction and solution of Markov reward models , 1996 .

[54]  P. Buchholz.  Exact and ordinary lumpability in finite Markov chains , 1994, Journal of Applied Probability.

[55]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[56]  John G. Kemeny,et al.  Finite Markov Chains. , 1960 .

[57]  Robert M. Marmorstein,et al.  Saturation Unbound , 2003, TACAS.

[58]  Lu Tian,et al.  On some equivalence relations for probabilistic processes , 1992, Fundamenta Informaticae.

[59]  C. Baier,et al.  Domain equations for probabilistic processes , 2000, Mathematical Structures in Computer Science.

[60]  William H. Sanders,et al.  Dependability Evaluation Using Composed SAN-Based Reward Models , 1992, J. Parallel Distributed Comput..

[61]  Peter Buchholz,et al.  Hierarchical Markovian Models: Symmetries and Reduction , 1995, Perform. Evaluation.

[62]  Gianfranco Ciardo,et al.  Storage Alternatives for Large Structured State Spaces , 1997, Computer Performance Evaluation.

[63]  Peter Kemper.  Numerical Analysis of Superposed GSPNs , 1996, IEEE Trans. Software Eng..

[64]  Peter Buchholz.  Exact Performance Equivalence: An Equivalence Relation for Stochastic Automata , 1999, Theor. Comput. Sci..

[65]  Boudewijn R. Haverkort,et al.  Specification techniques for Markov reward models , 1993, Discret. Event Dyn. Syst..

[66]  Jacob A. Abraham,et al.  A Numerical Technique for the Hierarchical Evaluation of Large, Closed Fault-Tolerant Systems , 1992 .

[67]  Peter Buchholz.  Lumpability and nearly-lumpability in hierarchical queueing networks , 1995, Proceedings of 1995 IEEE International Computer Performance and Dependability Symposium.

[68]  Edmundo de Souza e Silva,et al.  Calculating transient distributions of cumulative reward , 1995, SIGMETRICS '95/PERFORMANCE '95.

[69]  Mark Allen Weiss,et al.  Data structures and algorithm analysis in Ada , 1993 .

[70]  Gianfranco Ciardo,et al.  Efficient Reachability Set Generation and Storage Using Decision Diagrams , 1999, ICATPN.

[71]  Kishor S. Trivedi.  Probability and Statistics with Reliability, Queuing, and Computer Science Applications , 1984 .

[72]  Robert E. Tarjan,et al.  Self-adjusting binary search trees , 1985, JACM.

[73]  Peter Buchholz,et al.  Efficient Computation and Representation of Large Reachability Sets for Composed Automata , 2002, Discret. Event Dyn. Syst..

[74]  Stephen Gilmore,et al.  An Efficient Algorithm for Aggregating PEPA Models , 2001, IEEE Trans. Software Eng..

[75]  Alfred V. Aho,et al.  The Design and Analysis of Computer Algorithms , 1974 .

[76]  David R. Musser,et al.  STL tutorial and reference guide, second edition: C++ programming with the standard template library , 2001 .

[77]  Susanna Donatelli.  Superposed stochastic automata: a class of stochastic Petri nets amenable to parallel solution , 1991, Proceedings of the Fourth International Workshop on Petri Nets and Performance Models PNPM91.

[78]  Peter Buchholz,et al.  Numerical analysis of stochastic marked graph nets , 1995, Proceedings 6th International Workshop on Petri Nets and Performance Models.

[79]  Peter Buchholz,et al.  A Toolbox for Functional and Quantitative Analysis of DEDS , 1998, Computer Performance Evaluation.