The cyclic reduction algorithm: from Poisson equation to stochastic processes and beyond

Cyclic reduction is an algorithm invented by G. H. Golub and R. W. Hockney in the mid 1960s for solving linear systems related to the finite differences discretization of the Poisson equation over a rectangle. Among the algorithms of Gene Golub, it is one of the most versatile and powerful ever created. Recently, it has been applied to solve different problems from different applicative areas. In this paper we survey the main features of cyclic reduction, relate it to properties of analytic functions, recall its extension to solving more general finite and infinite linear systems, and different kinds of nonlinear matrix equations, including algebraic Riccati equations, with applications to Markov chains, queueing models and transport theory. Some new results concerning the convergence properties of cyclic reduction and its applicability are proved under very weak assumptions. New formulae for overcoming breakdown are provided.

[1]  Dario Andrea Bini,et al.  Polynomial factorization through Toeplitz matrix computations , 2003 .

[2]  Benny Van Houdt,et al.  Structured Markov chains solver: tool extension , 2009, VALUETOOLS.

[3]  Vaidyanathan Ramaswami,et al.  Introduction to Matrix Analytic Methods in Stochastic Modeling , 1999, ASA-SIAM Series on Statistics and Applied Mathematics.

[4]  L. Rogers.  Fluid Models in Queueing Theory and Wiener-Hopf Factorization of Markov Chains , 1994 .

[5]  Gene H. Golub,et al.  On direct methods for solving Poisson's equation , 1970, Milestones in Matrix Computation.

[6]  Nicholas J. Higham,et al.  Detecting and Solving Hyperbolic Quadratic Eigenvalue Problems , 2008, SIAM J. Matrix Anal. Appl..

[7]  Gene H. Golub,et al.  Cyclic Reduction - History and Applications , 1997 .

[8]  Alexandre Ostrowski.  Recherches sur la méthode de graeffe et les zéros des polynomes et des séries de laurent , 1940 .

[9]  S. Lennart Johnsson,et al.  Optimizing Tridiagonal Solvers for Alternating Direction Methods on Boolean Cube Multiprocessors , 1989, SIAM J. Sci. Comput..

[10]  Vaidyanathan Ramaswami,et al.  A logarithmic reduction algorithm for quasi-birth-death processes , 1993, Journal of Applied Probability.

[11]  Wen-Wei Lin,et al.  A structure-preserving doubling algorithm for nonsymmetric algebraic Riccati equation , 2006, Numerische Mathematik.

[12]  T. Politi,et al.  Parallel factorizations for tridiagonal matrices , 1993 .

[13]  BEATRICE MEINI,et al.  The Matrix Square Root from a New Functional Perspective: Theoretical Results and Computational Issues , 2005, SIAM J. Matrix Anal. Appl..

[14]  Frank D. Faulkner,et al.  Direct Solution of Elliptic Equations by Block Cyclic Reduction and Factorization , 1976 .

[15]  J. W. Eastwood,et al.  Book Review: Parallel computers: Architecture, programming and algorithms. R.W. Hockney and C.R. Jesshope, Adam-Hilger, Bristol, 1981. xii + 416 pages. £22.50 , 1982 .

[16]  Chun-Hua Guo,et al.  Efficient methods for solving a nonsymmetric algebraic Riccati equation arising in stochastic fluid models , 2006 .

[17]  Dario Bini,et al.  On Cyclic Reduction Applied to a Class of Toeplitz-Like Matrices Arising in Queueing Problems , 1995 .

[18]  David Williams,et al.  A ‘potential-theoretic’ note on the quadratic Wiener-Hopf equation for Q-matrices , 1982 .

[19]  B. Conolly.  Structured Stochastic Matrices of M/G/1 Type and Their Applications , 1991 .

[20]  Beatrice Meini,et al.  Non-skip-free M/G/1-type Markov chains and Laurent matrix power series , 2004 .

[21]  H. Schwandt,et al.  Truncated interval arithmetic block cyclic reduction , 1989 .

[22]  Clive Temperton.  On the FACR( l) algorithm for the discrete Poisson equation , 1980 .

[23]  Thomas Kailath,et al.  Fast reliable algorithms for matrices with structure , 1999 .

[24]  Martin A. Diamond,et al.  On a Cyclic Reduction Method for the Solution of Poisson’s Equations , 1976 .

[25]  L. Reichel.  The ordering of tridiagonal matrices in the cyclic reduction method for Poisson's equation , 1989 .

[26]  Paul N. author-Swarztrauber.  Approximate Cyclic Reduction for Solving Poisson's Equation , 1987 .

[27]  Attahiru Sule Alfa,et al.  Advances in matrix-analytic methods for stochastic models , 1998 .

[28]  Beatrice Meini,et al.  Numerical methods for structured Markov chains , 2005 .

[29]  San-qi Li,et al.  Analysis of multi-media traffic queues with finite buffer and overload control. I. Algorithm , 1991, IEEE INFCOM '91. The conference on Computer Communications. Tenth Annual Joint Comference of the IEEE Computer and Communications Societies Proceedings.

[30]  B. Meini,et al.  Structured Markov chains solver: software tools , 2006, SMCtools '06.

[31]  William L. Briggs,et al.  Fast Poisson solvers for MIMD computers , 1988, Parallel Comput..

[32]  R. Sweet.  A Cyclic Reduction Algorithm for Solving Block Tridiagonal Systems of Arbitrary Dimension , 1977 .

[33]  N. Higham.  Functions of Matrices: Theory and Computation (Other Titles in Applied Mathematics) , 2008 .

[34]  Plamen Y. Yalamov,et al.  Stability of the block cyclic reduction , 1996 .

[35]  Beatrice Meini,et al.  Approximate displacement rank and applications , 2001 .

[36]  Yousef Saad,et al.  A Parallel Block Cyclic Reduction Algorithm for the Fast Solution of Elliptic Equations , 1987, ICS.

[37]  Federico Poloni,et al.  From Algebraic Riccati equations to unilateral quadratic matrix equations: old and new algorithms , 2007, Numerical Methods for Structured Markov Chains.

[38]  Marcel F. Neuts,et al.  Matrix-Geometric Solutions in Stochastic Models , 1981 .

[39]  Guozhu Yao,et al.  Positive definite solution of the matrix equation $\boldsymbol {X=Q+A^{H}(I\otimes X-C)^{\delta}A}$ , 2011, Numerical Algorithms.

[40]  San-qi Li,et al.  Analysis of multimedia traffic queues with finite buffer and overload control II. Applications , 1992, [Proceedings] IEEE INFOCOM '92: The Conference on Computer Communications.

[41]  A. Ran,et al.  Necessary and sufficient conditions for the existence of a positive definite solution of the matrix equation X + A*X-1A = Q , 1993 .

[42]  Tuomo Rossi,et al.  A Parallel Fast Direct Solver for Block Tridiagonal Systems with Separable Matrices of Arbitrary Dimension , 1999, SIAM J. Sci. Comput..

[43]  G. Rodrigue,et al.  Preconditioning by incomplete block cyclic reduction , 1984 .

[44]  Peter Key,et al.  Teletraffic engineering in a competitive world : proceedings of the International Teletraffic Congress - ITC-16, Edinburgh International Conference Centre, United Kingdom, 7-11 June, 1999 , 1999 .

[45]  D. Heller.  Some Aspects of the Cyclic Reduction Algorithm for Block Tridiagonal Linear Systems , 1976 .

[46]  S. Steffé,et al.  Structured Markov chains solver: algorithms , 2006, SMCtools '06.

[47]  Beatrice Meini,et al.  Improved cyclic reduction for solving queueing problems , 1997, Numerical Algorithms.

[48]  J. Hillston.  Compositional Markovian Modelling Using a Process Algebra , 1995 .

[49]  Beatrice Meini,et al.  Effective Methods for Solving Banded Toeplitz Systems , 1999, SIAM J. Matrix Anal. Appl..

[50]  Tuomo Rossi,et al.  A Nonstandard Cyclic Reduction Method, Its Variants and Stability , 1999, SIAM J. Matrix Anal. Appl..

[51]  Roland A. Sweet,et al.  Algorithm 541: Efficient Fortran Subprograms for the Solution of Separable Elliptic Partial Differential Equations [D3] , 1979, TOMS.

[52]  Nicholas J. Higham,et al.  Iterative Solution of a Nonsymmetric Algebraic Riccati Equation , 2007, SIAM J. Matrix Anal. Appl..

[53]  R. Vandebril,et al.  Matrix Computations and Semiseparable Matrices , 2007 .

[54]  Wen-Wei Lin,et al.  Nonsymmetric Algebraic Riccati Equations and Hamiltonian-like Matrices , 1998, SIAM J. Matrix Anal. Appl..

[55]  Pierluigi Amodio.  Optimized cyclic reduction for the solution of linear tridiagonal systems on parallel computers , 1993 .

[56]  R. Sweet.  A Generalized Cyclic Reduction Algorithm , 1974 .

[57]  Chun-Hua Guo,et al.  On the Doubling Algorithm for a (Shifted) Nonsymmetric Algebraic Riccati Equation , 2007, SIAM J. Matrix Anal. Appl..

[58]  R. Sweet.  A Parallel and Vector Variant of the Cyclic Reduction Algorithm , 1988 .

[59]  Beatrice Meini,et al.  On the Solution of a Nonlinear Matrix Equation Arising in Queueing Problems , 1996, SIAM J. Matrix Anal. Appl..

[60]  W. Sun.  Cyclic reduction algorithm for solving collocation systems , 1996, Int. J. Comput. Math..

[61]  Jonq Juang.  Global Existence and Stability of Solutions of Matrix Riccati Equations , 2001 .

[62]  Arieh Iserles.  A First Course in the Numerical Analysis of Differential Equations: Fast Poisson solvers , 2008 .

[63]  Beatrice Meini,et al.  Solving matrix polynomial equations arising in queueing problems , 2002 .

[64]  G. Lotti.  A note on the solution of not balanced banded Toeplitz systems , 2007, Numer. Linear Algebra Appl..

[65]  Beatrice Meini,et al.  Effective Fast Algorithms for Polynomial Spectral Factorization , 2003, Numerical Algorithms.

[66]  Jacob Engwerda,et al.  Necessary and Sufficient Conditions for the Existence of a Positive Definite Solution of the Matrix Equation X + A*X - 'A = Q , 1993 .

[67]  P. Swarztrauber.  THE METHODS OF CYCLIC REDUCTION, FOURIER ANALYSIS AND THE FACR ALGORITHM FOR THE DISCRETE SOLUTION OF POISSON'S EQUATION ON A RECTANGLE* , 1977 .

[68]  Bernard Bialecki,et al.  Cyclic reduction and FACR methods for piecewise hermite bicubic orthogonal spline collocation , 1994, Numerical Algorithms.

[69]  W. Gander,et al.  Cyclic Reduction for Special Tridiagonal Systems , 1994 .

[70]  Beatrice Meini,et al.  M/G/1-TYPE MARKOV CHAINS , 2005 .

[71]  B. Iannazzo,et al.  A note on computing the matrix square root , 2003 .

[72]  Chun-Hua Guo,et al.  Convergence Analysis of the Latouche-Ramaswami Algorithm for Null Recurrent Quasi-Birth-Death Processes , 2001, SIAM J. Matrix Anal. Appl..

[73]  R. Vandebril,et al.  Matrix Computations and Semiseparable Matrices: Linear Systems , 2010 .

[74]  Chun-Hua Guo,et al.  Algorithms for hyperbolic quadratic eigenvalue problems , 2005, Math. Comput..

[75]  Federico Poloni,et al.  Fast solution of a certain Riccati equation through Cauchy-like matrices , 2009 .

[76]  P. Swarztrauber.  A direct Method for the Discrete Solution of Separable Elliptic Equations , 1974 .

[77]  Henry C. Thacher,et al.  Applied and Computational Complex Analysis. , 1988 .

[78]  Dario Bini,et al.  On the solution of algebraic Riccati equations arising in fluid queues , 2006 .

[79]  Beatrice Meini,et al.  Solving certain matrix equations by means of Toeplitz computations: algorithms and applications , 2001 .

[80]  Beatrice Meini,et al.  Factorization of analytic functions by means of Koenig's theorem and Toeplitz computations , 1998, Numerische Mathematik.

[81]  Bruno Iannazzo,et al.  Nonsymmetric algebraic Riccati equations associated with an M-matrix: recent advances and algorithms , 2007, Numerical Methods for Structured Markov Chains.

[82]  Roger W. Hockney,et al.  A Fast Direct Solution of Poisson's Equation Using Fourier Analysis , 1965, JACM.

[83]  P. Yalamov.  On the stability of the cyclic reduction without back substitution for tridiagonal systems , 1995 .

[84]  Noah H. Rhee,et al.  A quadratically convergent Bernoulli-like algorithm for solving matrix polynomial equations in Markov chains. , 2004 .

[85]  Marcin Paprzycki,et al.  A Cyclic Reduction Approach to the Numerical Solution of Boundary Value ODEs , 1997, SIAM J. Sci. Comput..

[86]  Stewart A. Levin,et al.  A Tricyclic Tridiagonal Equation Solver , 1992, SIAM J. Matrix Anal. Appl..

[87]  Ronald F. Boisvert,et al.  Algorithms for Special Tridiagonal Systems , 1991, SIAM J. Sci. Comput..

[88]  Beatrice Meini,et al.  Shift Techniques and Canonical Factorizations in the Solution of M/G/1-Type Markov Chains , 2005 .

[89]  Dario Bini,et al.  Computations with infinite Toeplitz matrices and polynomials , 2002 .

[90]  B. O. Anderson.  Second-order convergent algorithms for the steady-state Riccati equation , 1977, 1977 IEEE Conference on Decision and Control including the 16th Symposium on Adaptive Processes and A Special Symposium on Fuzzy Set Theory and Applications.

[91]  Nicola Mastronardi,et al.  A Parallel Version of the Cyclic Reduction Algorithm on a Hypercube , 1993, Parallel Comput..

[92]  Clive Temperton,et al.  Direct methods for the solution of the discrete Poisson equation: Some comparisons , 1979 .

[93]  Beatrice Meini,et al.  A probabilistic interpretation of cyclic reduction and its relationships with logarithmic reduction , 2008 .

[94]  Peter G. Taylor,et al.  Advances in Algorithmic Methods for Stochastic Models , 2000 .

[95]  Nicholas J. Higham,et al.  Algorithms for the matrix pth root , 2005, Numerical Algorithms.

[96]  Paul N. Swarztrauber,et al.  Vector and parallel methods for the direct solution of Poisson's equation , 1989 .

[97]  Beatrice Meini,et al.  Nonlinear matrix equations and structured linear algebra , 2006 .

[98]  Beatrice Meini,et al.  Analyzing M/G/1 paradigmsthrough QBDs: the role of the block structure in computing the matrixG , 2000 .

[99]  M. K. Kerimov,et al.  Applied and computational complex analysis. Vol. 1. Power series, integration, conformal mapping, location of zeros: Henrici P. xv + 682 pp., John Wiley and Sons, Inc., New York — London, 1974☆ , 1977 .

[100]  Nicholas J. Higham,et al.  Functions of matrices - theory and computation , 2008 .

[101]  Vaidyanathan Ramaswami,et al.  Matrix analytic methods for stochastic fluid flows , 1999 .

[102]  Thomas Kailath,et al.  Linear Systems , 1980 .

[103]  Beatrice Meini,et al.  Numerical Methods for Structured Markov Chains (Numerical Mathematics and Scientific Computation) , 2005 .

[104]  Pierluigi Amodio,et al.  Backward error analysis of cyclic reduction for the solution of tridiagonal systems , 1994 .

[105]  V. Olshevsky.  Structured Matrices in Mathematics, Computer Science, and Engineering II , 2001 .

[106]  R. Cottle.  Manifestations of the Schur complement , 1974 .

[107]  B. Levy,et al.  Hermitian solutions of the equation X = Q + NX−1N∗ , 1996 .

[108]  Hartmut Schwandt.  Cyclic reduction for tridiagonal systems of equations with interval coefficients on vector computers , 1989 .

[109]  Alexandre Ostrowski.  Addition à notre mémoire: ‘Recherches sur la méthode de graeffe et les zéros des polynômes et des séries de Laurent’ , 1942 .

[110]  Chun-Hua Guo.  Comments on a Shifted Cyclic Reduction Algorithm for Quasi-Birth-Death Problems , 2003, SIAM J. Matrix Anal. Appl..

[111]  W. Gautschi.  Orthogonal Polynomials: Computation and Approximation , 2004 .

[112]  Plamen Y. Yalamov,et al.  Stabilization by perturbation of ILL-conditioned cyclic reduction , 1998, Int. J. Comput. Math..

[113]  Beatrice Meini,et al.  Efficient computation of the extreme solutions of X + A*X-1A = Q and X - A*X-1A = Q , 2001, Math. Comput..