Structured analysis techniques for large Markov chains

Analysis of large and realistic Markov models suffers mainly from the so-called state space explosion. Large systems of equations have to be solved to obtain the result measures. The limiting factors of a solution are space and time. Apart from using more powerful computers for the solution, sophisticated data structures have been used for a compact storage of huge Markov chains and new efficient analysis techniques have been developed recently to exploit structure in the model for a more efficient analysis of the Markov chains. This paper gives an overview of analysis techniques based on model structure, describes the availability of these new techniques and outlines future research directions in the field.

[1]  Peter Buchholz,et al.  On the Exact and Approximate Analysis of Hierarchical Discrete Time Queueing Networks , 1995, MMB.

[2]  T. Y. WilliamJ,et al.  Numerical Methods in Markov Chain Modeling , 1992, Operational Research.

[3]  Peter Buchholz,et al.  Multilevel Solutions for Structured Markov Chains , 2000, SIAM J. Matrix Anal. Appl..

[4]  Brigitte Plateau,et al.  On the stochastic structure of parallelism and synchronization models for distributed algorithms , 1985, SIGMETRICS '85.

[5]  Holger Hermanns,et al.  On the use of MTBDDs for performability analysis and verification of stochastic systems , 2003, J. Log. Algebraic Methods Program..

[6]  William H. Sanders,et al.  The Möbius state-level abstract functional interface , 2002, Perform. Evaluation.

[7]  Alexander Bell.  Distributed Evaluation of Stochastic Petri nets , 2004, MMB.

[8]  Amy Nicole Langville,et al.  A Kronecker product approximate preconditioner for SANs , 2004, Numer. Linear Algebra Appl..

[9]  William H. Sanders,et al.  "On-the-fly" solution techniques for stochastic Petri nets and extensions , 1997, Proceedings of the Seventh International Workshop on Petri Nets and Performance Models.

[10]  Peter Buchholz,et al.  On generating a hierarchy for GSPN analysis , 1998, PERV.

[11]  David Parker,et al.  Symbolic Representations and Analysis of Large Probabilistic Systems , 2004, Validation of Stochastic Systems.

[12]  Markus Fischer,et al.  Parallele numerische Verfahren zur quantitativen Analyse logistischer Systeme , 2006 .

[13]  Peter Buchholz,et al.  Structured analysis approaches for large Markov chains , 1999 .

[14]  Gianfranco Ciardo,et al.  SMART: the stochastic model checking analyzer for reliability and timing , 2004, First International Conference on the Quantitative Evaluation of Systems, 2004. QEST 2004. Proceedings..

[15]  Peter Buchholz,et al.  Complexity of Memory-Efficient Kronecker Operations with Applications to the Solution of Markov Models , 2000, INFORMS J. Comput..

[16]  William L. Briggs,et al.  A multigrid tutorial , 1987 .

[17]  Christel Baier,et al.  Validation of Stochastic Systems: A Guide to Current Research (Lecture Notes in Computer Science) , 2004 .

[18]  Peter Buchholz,et al.  Hierarchical Structuring of Superposed GSPNs , 1999, IEEE Trans. Software Eng..

[19]  Gianfranco Ciardo,et al.  Storage Alternatives for Large Structured State Spaces , 1997, Computer Performance Evaluation.

[20]  Amy Nicole Langville,et al.  Testing the Nearest Kronecker Product Preconditioner on Markov Chains and Stochastic Automata Networks , 2004, INFORMS J. Comput..

[21]  Paulo Fernandes,et al.  Efficient descriptor-vector multiplications in stochastic automata networks , 1998, JACM.

[22]  PETER BUCHHOLZ,et al.  Block SOR Preconditioned Projection Methods for Kronecker Structured Markovian Representations , 2005, SIAM J. Sci. Comput..

[23]  Peter Buchholz.  Adaptive decomposition and approximation for the analysis of stochastic Petri nets , 2004, Perform. Evaluation.

[24]  Paulo Fernandes,et al.  The PEPS Software Tool , 2003, Computer Performance Evaluation / TOOLS.

[25]  Raymond H. Chan,et al.  Circulant preconditioners for stochastic automata networks , 2000, Numerische Mathematik.

[26]  Peter Buchholz,et al.  Block SOR for Kronecker structured representations , 2004 .

[27]  Randal E. Bryant,et al.  Graph-Based Algorithms for Boolean Function Manipulation , 1986, IEEE Transactions on Computers.

[28]  Gianfranco Ciardo,et al.  Using the exact state space of a Markov model to compute approximate stationary measures , 2000, SIGMETRICS '00.

[29]  Rashid Mehmood.  Serial Disk-based Analysis of Large Stochastic Models , 2004, Validation of Stochastic Systems.

[30]  William H. Sanders,et al.  An Efficient Disk-Based Tool for Solving Large Markov Models , 1998, Perform. Evaluation.

[31]  Peter Buchholz,et al.  A Toolbox for the Analysis of Discrete Event Dynamic Systems , 1999, CAV.

[32]  Peter Buchholz,et al.  Comparison of Multilevel Methods for Kronecker-based Markovian Representations , 2004, Computing.

[33]  Peter Buchholz,et al.  A class of hierarchical queueing networks and their analysis , 1994, Queueing Syst. Theory Appl..

[34]  Peter Buchholz,et al.  An adaptive aggregation/disaggregation algorithm for hierarchical Markovian models , 1999, Eur. J. Oper. Res..

[35]  Marta Z. Kwiatkowska,et al.  PRISM 2.0: a tool for probabilistic model checking , 2004, First International Conference on the Quantitative Evaluation of Systems, 2004. QEST 2004. Proceedings..

[36]  Manuel Silva Suárez,et al.  Structured Solution of Asynchronously Communicating Stochastic Modules , 1999, IEEE Trans. Software Eng..

[37]  Nieves R. Brisaboa,et al.  Approximate computation of transient results for large Markov chains , 2004 .

[38]  Tugrul Dayar,et al.  Iterative methods based on splittings for stochastic automata networks , 1998, Eur. J. Oper. Res..

[39]  Peter Buchholz,et al.  Kronecker Based Matrix Representations for Large Markov Models , 2004, Validation of Stochastic Systems.

[40]  Graham Horton,et al.  A multi-level solution algorithm for steady-state Markov chains , 1994, SIGMETRICS.

[41]  Tugrul Dayar,et al.  Comparison of Partitioning Techniques for Two-Level Iterative Solvers on Large, Sparse Markov Chains , 1999, SIAM J. Sci. Comput..

[42]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[43]  Peter Buchholz,et al.  On the Convergence of a Class of Multilevel Methods for Large Sparse Markov Chains , 2007, SIAM J. Matrix Anal. Appl..

[44]  William H. Sanders,et al.  Approximate computation of transient results for large Markov chains , 2004, First International Conference on the Quantitative Evaluation of Systems, 2004. QEST 2004. Proceedings..

[45]  Peter Buchholz,et al.  A Hierarchical View of GCSPNs and Its Impact on Qualitative and Quantitative Analysis , 1992, J. Parallel Distributed Comput..