Comparison of Multilevel Methods for Kronecker-based Markovian Representations

Abstract.The paper presents a class of numerical methods to compute the stationary distribution of Markov chains (MCs) with large and structured state spaces. A popular way of dealing with large state spaces in Markovian modeling and analysis is to employ Kronecker-based representations for the generator matrix and to exploit this matrix structure in numerical analysis methods. This paper presents various multilevel (ML) methods for a broad class of MCs with a hierarchcial Kronecker structure of the generator matrix. The particular ML methods are inspired by multigrid and aggregation-disaggregation techniques, and differ among each other by the type of multigrid cycle, the type of smoother, and the order of component aggregation they use. Numerical experiments demonstrate that so far ML methods with successive over-relaxation as smoother provide the most effective solvers for considerably large Markov chains modeled as HMMs with multiple macrostates.

[1]  Jean-Michel Fourneau,et al.  A Methodology for Solving Markov Models of Parallel Systems , 1991, J. Parallel Distributed Comput..

[2]  Peter Buchholz,et al.  A class of hierarchical queueing networks and their analysis , 1994, Queueing Syst. Theory Appl..

[3]  Paulo Fernandes,et al.  Efficient descriptor-vector multiplications in stochastic automata networks , 1998, JACM.

[4]  Brigitte Plateau.  On the stochastic structure of parallelism and synchronization models for distributed algorithms , 1985, SIGMETRICS 1985.

[5]  Susanna Donatelli,et al.  Structured Solution of Asynchronously , 1999 .

[6]  Wolfgang Hackbusch,et al.  Multi-grid methods and applications , 1985, Springer series in computational mathematics.

[7]  PETER BUCHHOLZ,et al.  Block SOR Preconditioned Projection Methods for Kronecker Structured Markovian Representations , 2005, SIAM J. Sci. Comput..

[8]  Henk A. van der Vorst,et al.  Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems , 1992, SIAM J. Sci. Comput..

[9]  Peter Buchholz,et al.  On generating a hierarchy for GSPN analysis , 1998, PERV.

[10]  Susanna Donatelli,et al.  Superposed Stochastic Automata: A Class of Stochastic Petri Nets with Parallel Solution and Distributed State Space , 1993, Perform. Evaluation.

[11]  Peter Buchholz,et al.  Multilevel Solutions for Structured Markov Chains , 2000, SIAM J. Matrix Anal. Appl..

[12]  Yao Li,et al.  Performance Petri net analysis of communications protocol software by delay-equivalent aggregation , 1991, Proceedings of the Fourth International Workshop on Petri Nets and Performance Models PNPM91.

[13]  Peter Buchholz,et al.  Complexity of Memory-Efficient Kronecker Operations with Applications to the Solution of Markov Models , 2000, INFORMS J. Comput..

[14]  Marco Ajmone Marsan,et al.  GSPN Models of Markovian Multiserver Multiqueue Systems , 1990, Perform. Evaluation.

[15]  Peter Buchholz,et al.  A Toolbox for Functional and Quantitative Analysis of DEDS , 1998, Computer Performance Evaluation.

[16]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[17]  P. Wesseling.  An Introduction to Multigrid Methods , 1992 .

[18]  Roland W. Freund,et al.  A Transpose-Free Quasi-Minimal Residual Algorithm for Non-Hermitian Linear Systems , 1993, SIAM J. Sci. Comput..

[19]  Peter Buchholz,et al.  Structured analysis approaches for large Markov chains , 1999 .

[20]  Peter Buchholz,et al.  Block SOR for Kronecker structured representations , 2004 .

[21]  Graham Horton,et al.  A multi-level solution algorithm for steady-state Markov chains , 1994, SIGMETRICS.

[22]  C. Loan.  The ubiquitous Kronecker product , 2000 .

[23]  Tugrul Dayar,et al.  Comparison of Partitioning Techniques for Two-Level Iterative Solvers on Large, Sparse Markov Chains , 1999, SIAM J. Sci. Comput..

[24]  I. Marek,et al.  Convergence analysis of an iterative aggregation/disaggregation method for computing stationary probability vectors of stochastic matrices , 1998, Numer. Linear Algebra Appl..

[25]  Tugrul Dayar,et al.  Iterative methods based on splittings for stochastic automata networks , 1998, Eur. J. Oper. Res..

[26]  Peter Kemper.  Numerical Analysis of Superposed GSPNs , 1996, IEEE Trans. Software Eng..

[27]  U. Krieger.  Numerical Solution of Large Finite Markov Chains by Algebraic Multigrid Techniques , 1995 .