On the Convergence of a Class of Multilevel Methods for Large Sparse Markov Chains

This paper investigates the theory behind the steady state analysis of large sparse Markov chains with a recently proposed class of multilevel methods using concepts from algebraic multigrid and iterative aggregation-disaggregation. The motivation is to better understand the convergence characteristics of the class of multilevel methods and to have a clearer formulation that will aid their implementation. In doing this, restriction (or aggregation) and prolongation (or disaggregation) operators of multigrid are used, and the Kronecker-based approach for hierarchical Markovian models is employed, since it suggests a natural and compact definition of grids (or levels). However, the formalism used to describe the class of multilevel methods for large sparse Markov chains has no influence on the theoretical results derived.

[1]  I. Marek,et al.  A note on local and global convergence analysis of iterative aggregation–disaggregation methods , 2006 .

[2]  Ivo Marek,et al.  Convergence analysis of an iterative aggregation/disaggregation method for computing stationary probability vectors of stochastic matrices , 1998, Numer. Linear Algebra Appl..

[3]  S. Serra,et al.  Multi-iterative methods , 1993 .

[4]  Peter Buchholz,et al.  Complexity of Memory-Efficient Kronecker Operations with Applications to the Solution of Markov Models , 2000, INFORMS J. Comput..

[5]  William L. Briggs,et al.  A multigrid tutorial , 1987 .

[6]  Richard Bellman,et al.  Introduction to Matrix Analysis , 1972 .

[7]  Richard Bellman,et al.  Introduction to matrix analysis (2nd ed.) , 1997 .

[8]  Peter Buchholz,et al.  A Toolbox for Functional and Quantitative Analysis of DEDS , 1998, Computer Performance Evaluation.

[9]  Michele Benzi,et al.  The Arithmetic Mean Method for Finding the Stationary Vector of Markov Chains , 1995, Parallel Algorithms Appl..

[10]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[11]  Graham Horton,et al.  A multi-level solution algorithm for steady-state Markov chains , 1994, SIGMETRICS.

[12]  C. Loan.  The ubiquitous Kronecker product , 2000 .

[13]  Paulo Fernandes,et al.  Optimizing tensor product computations in stochastic automata networks , 1998 .

[14]  Peter Buchholz,et al.  On generating a hierarchy for GSPN analysis , 1998, PERV.

[15]  I. Marek,et al.  Convergence analysis of an iterative aggregation/disaggregation method for computing stationary probability vectors of stochastic matrices , 1998, Numer. Linear Algebra Appl..

[16]  J. W. Ruge,et al.  4. Algebraic Multigrid , 1987 .

[17]  J. Gilbert.  Predicting Structure in Sparse Matrix Computations , 1994 .

[18]  U. Krieger.  Numerical Solution of Large Finite Markov Chains by Algebraic Multigrid Techniques , 1995 .

[19]  G. W. Stewart,et al.  Matrix algorithms , 1998 .

[20]  Robert J. Plemmons,et al.  Nonnegative Matrices in the Mathematical Sciences , 1979, Classics in Applied Mathematics.

[21]  P. Wesseling.  An Introduction to Multigrid Methods , 1992 .

[22]  J. Paul Tremblay,et al.  Discrete Mathematical Structures with Applications to Computer Science , 1975 .

[23]  Peter Buchholz,et al.  Comparison of Multilevel Methods for Kronecker-based Markovian Representations , 2004, Computing.

[24]  Peter Buchholz,et al.  Structured analysis approaches for large Markov chains , 1999 .

[25]  Anne Greenbaum,et al.  Analysis of a Multigrid Method as an Iterative Technique for Solving Linear Systems , 1984 .

[26]  Peter Buchholz,et al.  Multilevel Solutions for Structured Markov Chains , 2000, SIAM J. Matrix Anal. Appl..

[27]  Peter Buchholz,et al.  Block SOR for Kronecker structured representations , 2004 .

[28]  Paulo Fernandes,et al.  Efficient descriptor-vector multiplications in stochastic automata networks , 1998, JACM.

[29]  PETER BUCHHOLZ,et al.  Block SOR Preconditioned Projection Methods for Kronecker Structured Markovian Representations , 2005, SIAM J. Sci. Comput..

[30]  Peter Buchholz,et al.  Hierarchical structuring of superposed GSPNs , 1997, Proceedings of the Seventh International Workshop on Petri Nets and Performance Models.

[31]  A. Borobia,et al.  On nonnegative matrices similar to positive matrices , 1997 .

[32]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[33]  Tugrul Dayar,et al.  Iterative methods based on splittings for stochastic automata networks , 1998, Eur. J. Oper. Res..

[34]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .