The Use of Parallel Polynomial Preconditioners in the Solution of Systems of Linear Equations

This thesis mainly explores the use of polynomial preconditioners in iterative solvers for large-scale sparse linear systems Ax = b. It is well known that preconditioners can significantly improve the convergence of solvers, particularly when the coefficient matrix is ill-conditioned. Further, polynomial preconditioners have several advantages over other popular preconditioners — they may be implemented easily, they are highly parallel, and they are extremely agile. Due to the intrinsic disadvantages of polynomial methods (e.g., spectrum information is needed, poor stability of the large-degree polynomial preconditioning) and the limitation of computing technologies, the polynomial preconditioning technique was somehow ignored in the past ten years. Fortunately, at present, polynomial preconditioners are attracting more and more attention with the development of computer science. The construction of appropriate polynomial preconditioners can be transformed into constrained optimization problem, namely that of finding am-degree polynomial in matrix A, Pm(A), such that Pm(A) ≈ A. Three typical polynomial preconditioners arise, Neumann-series, Least-squares and Minimax. In this thesis Symmetric linear equations will be mainly discussed, where the spectrum of the coefficient matrix is expressed by a real compact set. In the case of SID (Symmetric Indefinite) linear systems, only Least-squares and Chebyshev methods are applicable. The implementation and detailed performance analysis of the Generalized Least-squares and the Generalized Minimax preconditioners are the main effort of this work. The convergence property of polynomial preconditioned iterative solvers is closely dependent on an accurate estimate of the eigenspectrum of the coefficient matrix. Consequently, it is natural to combine polynomial preconditioners with GMRES, the well-known standard algorithm for the computation of the solution of systems of linear equations which incorporates the so-called Arnoldi process. Thus, the use

[1]  Rene F. Swarttouw,et al.  Orthogonal Polynomials , 2005, Series and Products in the Development of Mathematics.

[2]  Claude Brezinski,et al.  Avoiding breakdown in the CGS algorithm , 1991, Numerical Algorithms.

[3]  Claude Brezinski,et al.  Avoiding breakdown and near-breakdown in Lanczos type algorithms , 1991, Numerical Algorithms.

[4]  Anne Greenbaum,et al.  Approximating the inverse of a matrix for use in iterative algorithms on vector processors , 1979, Computing.

[5]  Kumar K. Tamma,et al.  A‐scalability and an integrated computational technology and framework for non‐linear structural dynamics. Part 1: Theoretical developments and parallel formulations , 2003 .

[6]  K. Tamma,et al.  A variationally consistent framework for the design of integrator and updates of generalized single step representations for structural dynamics , 2003 .

[7]  Masha Sosonkina,et al.  pARMS: a parallel version of the algebraic recursive multilevel solver , 2003, Numer. Linear Algebra Appl..

[8]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[9]  Marek Szularz,et al.  Generalized least-squares polynomial preconditioners for symmetric indefinite linear equations , 2002, Parallel Comput..

[10]  Olof B. Widlund,et al.  DUAL-PRIMAL FETI METHODS FOR THREE-DIMENSIONAL ELLIPTIC PROBLEMS WITH HETEROGENEOUS COEFFICIENTS , 2022 .

[11]  Marek Szularz,et al.  Polynomial Preconditioning for Specially Structured Linear Systems of Equations , 2001, Euro-Par.

[12]  D. Rixen,et al.  FETI‐DP: a dual–primal unified FETI method—part I: A faster alternative to the two‐level FETI method , 2001 .

[13]  Jan Mandel,et al.  On the convergence of a dual-primal substructuring method , 2000, Numerische Mathematik.

[14]  C. Farhat,et al.  A scalable dual-primal domain decomposition method , 2000, Numer. Linear Algebra Appl..

[15]  Kumar K. Tamma,et al.  A unified family of generalized integration operators [GInO] for non-linear structural dynamics: implementation aspects , 2000 .

[16]  Olof B. Widlund,et al.  A Domain Decomposition Method with Lagrange Multipliers and Inexact Solvers for Linear Elasticity , 2000, SIAM J. Sci. Comput..

[17]  Gundolf Haase A PARALLEL AMG FOR OVERLAPPING AND NON-OVERLAPPING DOMAIN DECOMPOSITION , 2000 .

[18]  Charbel Farhat,et al.  A family of domain decomposition methods for the massively parallel solution of computational mechanics problems , 2000 .

[19]  Kumar K. Tamma,et al.  Highly scalable parallel computational models for large-scale RTM process modeling simulations, part 2 : Parallel formulation theory and implementation , 1999 .

[20]  G. Meurant Computer Solution of Large Linear Systems , 1999 .

[21]  Maurice Clint,et al.  Explicitly restarted Lanczos algorithms in an MPP environment , 1999, Parallel Comput..

[22]  Jack J. Dongarra,et al.  A Parallel Divide and Conquer Algorithm for the Symmetric Eigenvalue Problem on Distributed Memory Architectures , 1999, SIAM J. Sci. Comput..

[23]  Marcus J. Grote,et al.  A Block Version of the SPAI Preconditioner , 1999, PP.

[24]  S. SIAMJ. A PARALLEL DIVIDE AND CONQUER ALGORITHM FOR THE SYMMETRIC EIGENVALUE PROBLEM ON DISTRIBUTED MEMORY ARCHITECTURES , 1999 .

[25]  Edmond Chow,et al.  Preserving Symmetry in Preconditioned Krylov Subspace Methods , 1998, SIAM J. Sci. Comput..

[26]  Gene H. Golub,et al.  Adaptively Preconditioned GMRES Algorithms , 1998, SIAM J. Sci. Comput..

[27]  Victor Eijkhout Overview of Iterative Linear System Solver Packages , 1998 .

[28]  Charbel Farhat,et al.  A unified framework for accelerating the convergence of iterative substructuring methods with Lagrange multipliers , 1998 .

[29]  H. Wozniakowski,et al.  Estimating a largest eigenvector by Lanczos and polynomial algorithms with a random start , 1998 .

[30]  C. Farhat,et al.  The two-level FETI method. Part II: Extension to shell problems, parallel implementation and performance results , 1998 .

[31]  C. Farhat,et al.  The two-level FETI method for static and dynamic plate problems Part I: An optimal iterative solver for biharmonic systems , 1998 .

[32]  N. Gould,et al.  Sparse Approximate-Inverse Preconditioners Using Norm-Minimization Techniques , 1998, SIAM J. Sci. Comput..

[33]  Kevin Burrage,et al.  On the performance of various adaptive preconditioned GMRES strategies , 1998, Numer. Linear Algebra Appl..

[34]  S. J. Sci PRESERVING SYMMETRY IN PRECONDITIONED KRYLOV SUBSPACE METHODS , 1998 .

[35]  Martyn R. Field Optimizing a Parallel Conjugate Gradient Solver , 1998, SIAM J. Sci. Comput..

[36]  Andreas Frommer,et al.  Restarted GMRES for Shifted Linear Systems , 1998, SIAM J. Sci. Comput..

[37]  Rakesh K. Kapania,et al.  A new adaptive GMRES algorithm for achieving high accuracy , 1998, Numer. Linear Algebra Appl..

[38]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[39]  Marcus J. Grote,et al.  Parallel Preconditioning with Sparse Approximate Inverses , 1997, SIAM J. Sci. Comput..

[40]  S. A. STOTLAND,et al.  Orderings for Parallel Conjugate Gradient Preconditioners , 1997, SIAM J. Sci. Comput..

[41]  K. Meerbergen,et al.  The Restarted Arnoldi Method Applied to Iterative Linear System Solvers for the Computation of Rightmost Eigenvalues , 1997 .

[42]  H. Walker,et al.  GMRES On (Nearly) Singular Systems , 1997, SIAM J. Matrix Anal. Appl..

[43]  Kim-Chuan Toh,et al.  GMRES vs. Ideal GMRES , 1997, SIAM J. Matrix Anal. Appl..

[44]  Daniel Rixen,et al.  Preconditioning the FETI Method for Problems with Intra- and Inter-Subdomain Coefficient Jumps , 1997 .

[45]  Thomas A. Manteuffel,et al.  Minimal Residual Method Stronger than Polynomial Preconditioning , 1996, SIAM J. Matrix Anal. Appl..

[46]  B. Fischer Polynomial Based Iteration Methods for Symmetric Linear Systems , 1996 .

[47]  Sosonkina Maria,et al.  A New Adaptive GMRES Algorithm for Achieving High Accuracy , 1996 .

[48]  Yousef Saad,et al.  Overlapping Domain Decomposition Algorithms for General Sparse Matrices , 1996, Numer. Linear Algebra Appl..

[49]  Åke Björck,et al.  Numerical methods for least square problems , 1996 .

[50]  Anne Greenbaum,et al.  Relations between Galerkin and Norm-Minimizing Iterative Methods for Solving Linear Systems , 1996, SIAM J. Matrix Anal. Appl..

[51]  Jack Dongarra,et al.  MPI: The Complete Reference , 1996 .

[52]  J. Navarro-Pedreño Numerical Methods for Least Squares Problems , 1996 .

[53]  Richard Y. Kain Advanced computer architecture: a system design approach , 1996, WCAE-2 '96.

[54]  C. Farhat,et al.  A scalable Lagrange multiplier based domain decomposition method for time‐dependent problems , 1995 .

[55]  Adhemar Bultheel,et al.  Vector Orthogonal Polynomials and Least Squares Approximation , 1995, SIAM J. Matrix Anal. Appl..

[56]  K. Bathe Finite Element Procedures , 1995 .

[57]  A. Bruaset A survey of preconditioned iterative methods , 1995 .

[58]  Vipin Kumar,et al.  Performance and Scalability of Preconditioned Conjugate Gradient Methods on Parallel Computers , 1995, IEEE Trans. Parallel Distributed Syst..

[59]  Zhaojun Bai,et al.  Progress in the numerical solution of the nonsymmetric eigenvalue problem , 1995, Numer. Linear Algebra Appl..

[60]  Henk A. van der Vorst,et al.  Approximate solutions and eigenvalue bounds from Krylov subspaces , 1995, Numer. Linear Algebra Appl..

[61]  S. Eisenstat,et al.  A Stable and Efficient Algorithm for the Rank-One Modification of the Symmetric Eigenproblem , 1994, SIAM J. Matrix Anal. Appl..

[62]  D. Calvetti,et al.  Application of a block modified Chebyshev algorithm to the iterative solution of symmetric linear systems with multiple right hand side vectors , 1994 .

[63]  Wayne Joubert,et al.  A Robust GMRES-Based Adaptive Polynomial Preconditioning Algorithm for Nonsymmetric Linear Systems , 1994, SIAM J. Sci. Comput..

[64]  John N. Shadid,et al.  A Comparison of Preconditioned Nonsymmetric Krylov Methods on a Large-Scale MIMD Machine , 1994, SIAM J. Sci. Comput..

[65]  Roland W. Freund,et al.  On Adaptive Weighted Polynomial Preconditioning for Hermitian Positive Definite Matrices , 1994, SIAM J. Sci. Comput..

[66]  Gene H. Golub,et al.  An adaptive Chebyshev iterative method\newline for nonsymmetric linear systems based on modified moments , 1994 .

[67]  George Karypis,et al.  Introduction to Parallel Computing , 1994 .

[68]  Youcef Saad,et al.  Highly Parallel Preconditioners for General Sparse Matrices , 1994 .

[69]  Richard Barrett,et al.  Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods , 1994, Other Titles in Applied Mathematics.

[70]  Roland W. Freund,et al.  A Transpose-Free Quasi-Minimal Residual Algorithm for Non-Hermitian Linear Systems , 1993, SIAM J. Sci. Comput..

[71]  G. Golub,et al.  How to generate unknown orthogonal polynomials out of known orthogonal polynomials , 1992 .

[72]  John N. Shadid,et al.  Sparse iterative algorithm software for large-scale MIMD machines: An initial discussion and implementation , 1992, Concurr. Pract. Exp..

[73]  Henk A. van der Vorst,et al.  Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems , 1992, SIAM J. Sci. Comput..

[74]  Hesham El-Rewini,et al.  Introduction to Parallel Computing , 1992 .

[75]  R. Freund,et al.  QMR: a quasi-minimal residual method for non-Hermitian linear systems , 1991 .

[76]  C. Farhat,et al.  A method of finite element tearing and interconnecting and its parallel solution algorithm , 1991 .

[77]  S. Ashby Minimax polynomial preconditioning for Hermitian linear systems , 1991 .

[78]  Charbel Farhat,et al.  A Lagrange multiplier based divide and conquer finite element algorithm , 1991 .

[79]  David S. Watkins,et al.  Fundamentals of matrix computations , 1991 .

[80]  Tony F. Chan,et al.  A Note on the Efficiency of Domain Decomposed Incomplete Factorizations , 1990, SIAM J. Sci. Comput..

[81]  I. Duff,et al.  The effect of ordering on preconditioned conjugate gradients , 1989 .

[82]  H. V. D. Vorst,et al.  High Performance Preconditioning , 1989 .

[83]  Gene H. Golub,et al.  On generating polynomials which are orthogonal over several intervals , 1989 .

[84]  P. Sonneveld CGS, A Fast Lanczos-Type Solver for Nonsymmetric Linear systems , 1989 .

[85]  Angel L. DeCegama,et al.  The technology of parallel processing: parallel processing architectures and VLSI hardware (vol. 1) , 1989 .

[86]  S. Ashby Polynomial Preconditioning for Conjugate Gradient Methods , 1988 .

[87]  Gilbert Strang,et al.  Introduction to applied mathematics , 1988 .

[88]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[89]  K. Law A parallel finite element solution method , 1986 .

[90]  Y. Saad,et al.  Practical Use of Polynomial Preconditionings for the Conjugate Gradient Method , 1985 .

[91]  Wolfgang Hackbusch,et al.  Multi-grid methods and applications , 1985, Springer series in computational mathematics.

[92]  Zhishun A. Liu,et al.  A Look Ahead Lanczos Algorithm for Unsymmetric Matrices , 1985 .

[93]  T. Manteuffel,et al.  Necessary and Sufficient Conditions for the Existence of a Conjugate Gradient Method , 1984 .

[94]  Y. Saad,et al.  Iterative Solution of Indefinite Symmetric Linear Systems by Methods Using Orthogonal Polynomials over Two Disjoint Intervals , 1983 .

[95]  J. Dixon Estimating Extremal Eigenvalues and Condition Numbers of Matrices , 1983 .

[96]  C. Micchelli,et al.  Polynomial Preconditioners for Conjugate Gradient Calculations , 1983 .

[97]  W. Gautschi On Generating Orthogonal Polynomials , 1982 .

[98]  Henk A. van der Vorst,et al.  A Vectorizable Variant of some ICCG Methods , 1982 .

[99]  H. V. D. Vorst,et al.  Iterative solution methods for certain sparse linear systems with a non-symmetric matrix arising from PDE-problems☆ , 1981 .

[100]  B. Parlett The Symmetric Eigenvalue Problem , 1981 .

[101]  Charles L. Lawson,et al.  Basic Linear Algebra Subprograms for Fortran Usage , 1979, TOMS.

[102]  J. H. Wilkinson,et al.  AN ESTIMATE FOR THE CONDITION NUMBER OF A MATRIX , 1979 .

[103]  J. Bunch,et al.  Rank-one modification of the symmetric eigenproblem , 1978 .

[104]  J. Meijerink,et al.  An iterative solution method for linear systems of which the coefficient matrix is a symmetric -matrix , 1977 .

[105]  M. Saunders,et al.  Solution of Sparse Indefinite Systems of Linear Equations , 1975 .

[106]  Murray R. Spiegel Scaum's outline of theory and problems of vector analysis : and an introduction to tensor analysis S 1 (metric) edition / by Murray R. Spiegel , 1974 .

[107]  K. Harbarth K. Rektorys, Survey of Applicable Mathematics. 1369 S. m. Fig. London 1969. Iliffe Books Ltd. Preis geb. 85 s. net , 1973 .

[108]  Louis A. Hageman,et al.  Iterative Solution of Large Linear Systems. , 1971 .

[109]  T. Broadbent,et al.  Survey of Applicable Mathematics , 1970, The Mathematical Gazette.

[110]  H. Meschkowski Series Expansions for Mathematical Physicists , 1969 .

[111]  D. C. Handscomb,et al.  Methods of Numerical Approximation , 1967 .

[112]  B. Wendroff Theoretical Numerical Analysis , 1966 .

[113]  E. Cheney Introduction to approximation theory , 1966 .

[114]  J. H. Wilkinson The algebraic eigenvalue problem , 1966 .

[115]  P. Davis Interpolation and approximation , 1965 .

[116]  F. B. Hildebrand,et al.  Introduction To Numerical Analysis , 1957 .

[117]  C. Lanczos Chebyshev polynomials in the solution of large-scale linear systems , 1952, ACM '52.