Iterant recombination with one-norm minimization for multilevel Markov chain algorithms via the ellipsoid method

Recently, it was shown how the convergence of a class of multigrid methods for computing the stationary distribution of sparse, irreducible Markov chains can be accelerated by the addition of an outer iteration based on iterant recombination. The acceleration was performed by selecting a linear combination of previous fine-level iterates with probability constraints to minimize the two-norm of the residual using a quadratic programming method. In this paper we investigate the alternative of minimizing the one-norm of the residual. This gives rise to a nonlinear convex program which must be solved at each acceleration step. To solve this minimization problem we propose to use a deep-cuts ellipsoid method for nonlinear convex programs. The main purpose of this paper is to investigate whether an iterant recombination approach can be obtained in this way that is competitive in terms of execution time and robustness. We derive formulas for subgradients of the one-norm objective function and the constraint functions, and show how an initial ellipsoid can be constructed that is guaranteed to contain the exact solution and give conditions for its existence. We also investigate using the ellipsoid method to minimize the two-norm. Numerical tests show that the one-norm and two-norm acceleration procedures yield a similar reduction in the number of multigrid cycles. The tests also indicate that one-norm ellipsoid acceleration is competitive with two-norm quadratic programming acceleration in terms of running time with improved robustness.

[1]  T. Y. WilliamJ,et al.  Numerical Methods in Markov Chain Modeling , 1992, Operational Research.

[2]  Herbert A. Simon,et al.  Aggregation of Variables in Dynamic Systems , 1961 .

[3]  Michael Kupferschmid,et al.  Using deep cuts in an ellipsoid algorithm for nonlinear programming , 1985 .

[4]  S. McCormick,et al.  A multigrid tutorial (2nd ed.) , 2000 .

[5]  J. Mandel,et al.  A local convergence proof for the iterative aggregation method , 1983 .

[6]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[7]  I. Marek,et al.  Convergence analysis of an iterative aggregation/disaggregation method for computing stationary probability vectors of stochastic matrices , 1998, Numer. Linear Algebra Appl..

[8]  William J. Stewart,et al.  Numerical Solution of Markov Chains , 1993 .

[9]  Thomas A. Manteuffel,et al.  Multilevel Adaptive Aggregation for Markov Chains, with Application to Web Ranking , 2008, SIAM J. Sci. Comput..

[10]  Hans De Sterck,et al.  Recursively Accelerated Multilevel Aggregation for Markov Chains , 2010, SIAM J. Sci. Comput..

[11]  Irad Yavneh,et al.  On-the-Fly Adaptive Smoothed Aggregation Multigrid for Markov Chains , 2011, SIAM J. Sci. Comput..

[12]  L. G. H. Cijan A polynomial algorithm in linear programming , 1979 .

[13]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[14]  William L. Briggs,et al.  A multigrid tutorial, Second Edition , 2000 .

[15]  Michael J. Todd,et al.  Modifications and implementation of the ellipsoid algorithm for linear programming , 1982, Math. Program..

[16]  John N. Tsitsiklis,et al.  Introduction to linear optimization , 1997, Athena scientific optimization and computation series.

[17]  N. Z. Shor Cut-off method with space extension in convex programming problems , 1977, Cybernetics.

[18]  Michael J. Todd,et al.  The Ellipsoid Method: A Survey , 1980 .

[19]  W. Miranker,et al.  Acceleration by aggregation of successive approximation methods , 1982 .

[20]  W. Greub Linear Algebra , 1981 .

[21]  William L. Briggs,et al.  A multigrid tutorial , 1987 .

[22]  Hector A. Rosales-Macedo Nonlinear Programming: Theory and Algorithms (2nd Edition) , 1993 .

[23]  Michael J. Todd,et al.  Feature Article - The Ellipsoid Method: A Survey , 1981, Oper. Res..

[24]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms , 1993 .

[25]  Thomas A. Manteuffel,et al.  Algebraic Multigrid for Markov Chains , 2010, SIAM J. Sci. Comput..

[26]  Peter Buchholz,et al.  Multilevel Solutions for Structured Markov Chains , 2000, SIAM J. Matrix Anal. Appl..

[27]  L. Khachiyan Polynomial algorithms in linear programming , 1980 .

[28]  Hans-Jakob Lüthi,et al.  On the Solution of Variational Inequalities by the Ellipsoid Method , 1985, Math. Oper. Res..

[29]  S. Leutenegger,et al.  ON THE UTILITY OF THE MULTI-LEVEL ALGORITHM FOR THE SOLUTION OF NEARLY COMPLETELY DECOMPOSABLE MARKOV CHAINS , 1994 .

[30]  Thomas A. Manteuffel,et al.  Smoothed Aggregation Multigrid for Markov Chains , 2010, SIAM J. Sci. Comput..

[31]  Michael Kupferschmid,et al.  An ellipsoid algorithm for nonlinear programming , 1983, Math. Program..

[32]  Irad Yavneh,et al.  Square and stretch multigrid for stochastic matrix eigenproblems , 2010, Numer. Linear Algebra Appl..

[33]  W. Stewart,et al.  ITERATIVE METHODS FOR COMPUTING STATIONARY DISTRIBUTIONS OF NEARLY COMPLETELY DECOMPOSABLE MARKOV CHAINS , 1984 .

[34]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[35]  Saudi Arabia,et al.  Aggregation/Disaggregation Methods for Computing the Stationary Distribution of Markov Chains with Application to Multiprogramming System , 1994 .

[36]  Michael K. Molloy Performance Analysis Using Stochastic Petri Nets , 1982, IEEE Transactions on Computers.

[37]  Udo R. Krieger,et al.  On a two-level multigrid solution method for finite Markov chains , 1995 .

[38]  Thomas A. Manteuffel,et al.  Top-level acceleration of adaptive algebraic multilevel methods for steady-state solution to Markov chains , 2011, Adv. Comput. Math..

[39]  William J. Stewart,et al.  Iterative aggregation/disaggregation techniques for nearly uncoupled markov chains , 1985, JACM.

[40]  I. Marek,et al.  Convergence theory of some classes of iterative aggregation/disaggregation methods for computing stationary probability vectors of stochastic matrices , 2003 .

[41]  Graham Horton,et al.  A multi-level solution algorithm for steady-state Markov chains , 1994, SIGMETRICS.

[42]  Winfried K. Grassmann,et al.  Regenerative Analysis and Steady State Distributions for Markov Chains , 1985, Oper. Res..

[43]  Falko Bause,et al.  Stochastic Petri Nets , 1996 .

[44]  Achi Brandt,et al.  On Recombining Iterants in Multigrid Algorithms and Problems with Small Islands , 1995, SIAM J. Sci. Comput..

[45]  U. Krieger Numerical Solution of Large Finite Markov Chains by Algebraic Multigrid Techniques , 1995 .

[46]  J. B. G. Frenk,et al.  A deep cut ellipsoid algorithm for convex programming: Theory and applications , 1994, Math. Program..

[47]  Robert J. Plemmons,et al.  Nonnegative Matrices in the Mathematical Sciences , 1979, Classics in Applied Mathematics.

[48]  Cornelis W. Oosterlee,et al.  KRYLOV SUBSPACE ACCELERATION FOR NONLINEAR MULTIGRID SCHEMES , 1997 .