Preconditioning

The computational solution of problems can be restricted by the availability of solution methods for linear(ized) systems of equations. In conjunction with iterative methods, preconditioning is often the vital component in enabling the solution of such systems when the dimension is large. We attempt a broad review of preconditioning methods.

[1]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[2]  M. Benzi.  Preconditioning techniques for large linear systems: a survey , 2002 .

[3]  R. Freund,et al.  QMR: a quasi-minimal residual method for non-Hermitian linear systems , 1991 .

[4]  Andrew J. Wathen,et al.  A Preconditioned MINRES Method for Nonsymmetric Toeplitz Matrices , 2015, SIAM J. Matrix Anal. Appl..

[5]  Tingzhu Huang,et al.  New Block Triangular Preconditioners for Saddle Point Linear Systems with Highly Singular (1,1) Blocks , 2009 .

[6]  Sander Rhebergen,et al.  Analysis of Block Preconditioners for Models of Coupled Magma/Mantle Dynamics , 2013, SIAM J. Sci. Comput..

[7]  A. Wathen,et al.  The convergence rate of the minimal residual method for the Stokes problem , 1995 .

[8]  M. Benzi,et al.  Some Preconditioning Techniques for Saddle Point Problems , 2008 .

[9]  Alle-Jan van der Veen,et al.  Some Fast Algorithms for Sequentially Semiseparable Representations , 2005, SIAM J. Matrix Anal. Appl..

[10]  Wei-Pai Tang,et al.  Sparse Approximate Inverse Smoother for Multigrid , 2000, SIAM J. Matrix Anal. Appl..

[11]  Harry Yserentant,et al.  On the multi-level splitting of finite element spaces , 1986 .

[12]  Luca Bergamaschi,et al.  Preconditioning Indefinite Systems in Interior Point Methods for Optimization , 2004, Comput. Optim. Appl..

[13]  Howard C. Elman,et al.  IFISS: A Computational Laboratory for Investigating Incompressible Flow Problems , 2014, SIAM Rev..

[14]  Patrick Amestoy,et al.  MUMPS : A General Purpose Distributed Memory Sparse Solver , 2000, PARA.

[15]  Jinchao Xu,et al.  Nodal Auxiliary Space Preconditioning in H(curl) and H(div) Spaces , 2007, SIAM J. Numer. Anal..

[16]  J. Korzak.  Eigenvalue Relations and Conditions of Matrices Arising in Linear Programming , 1999, Computing.

[17]  Henk A. van der Vorst,et al.  Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems , 1992, SIAM J. Sci. Comput..

[18]  Cornelis Vuik,et al.  Spectral Analysis of the Discrete Helmholtz Operator Preconditioned with a Shifted Laplacian , 2007, SIAM J. Sci. Comput..

[19]  A. Wathen,et al.  Minimum residual methods for augmented systems , 1998 .

[20]  W. Hackbusch,et al.  On H2-Matrices , 2000 .

[21]  Barry F. Smith,et al.  Domain Decomposition: Parallel Multilevel Methods for Elliptic Partial Differential Equations , 1996 .

[22]  D. R. Fokkema,et al.  BiCGstab(ell) for Linear Equations involving Unsymmetric Matrices with Complex Spectrum , 1993 .

[23]  J. Scott,et al.  HSL_MI20: An efficient AMG preconditioner for finite element problems in 3D , 2010 .

[24]  Kent-André Mardal,et al.  Preconditioning discretizations of systems of partial differential equations , 2011, Numer. Linear Algebra Appl..

[25]  B. Fischer.  Polynomial Based Iteration Methods for Symmetric Linear Systems , 1996 .

[26]  Andrew J. Wathen,et al.  Analysis of Preconditioners for Saddle-Point Problems , 2004, SIAM J. Sci. Comput..

[27]  Barry Lee,et al.  Finite elements and fast iterative solvers: with applications in incompressible fluid dynamics , 2006, Math. Comput..

[28]  Eric de Sturler,et al.  Recycling Krylov Subspaces for Sequences of Linear Systems , 2006, SIAM J. Sci. Comput..

[29]  Axel Klawonn,et al.  Block triangular preconditioners for nonsymmetric saddle point problems: field-of-values analysis , 1999, Numerische Mathematik.

[30]  Andrew J. Wathen,et al.  Natural Preconditioning and Iterative Methods for Saddle Point Systems , 2015, SIAM Rev..

[31]  G. Meurant.  Computer Solution of Large Linear Systems , 1999 .

[32]  Andrew J. Wathen,et al.  Combination preconditioning of saddle point systems for positive definiteness , 2013, Numer. Linear Algebra Appl..

[33]  Martin Stoll,et al.  Combination Preconditioning and the Bramble-Pasciak+ Preconditioner , 2008, SIAM J. Matrix Anal. Appl..

[34]  R. S. Falk,et al.  PRECONDITIONING IN H (div) AND APPLICATIONS , 1997 .

[35]  Gene H. Golub,et al.  A generalized conjugate gradient method for non-symmetric systems of linear equations , 2007, Milestones in Matrix Computation.

[36]  I. Gustafsson.  A class of first order factorization methods , 1978 .

[37]  Olga Taussky,et al.  The role of symmetric matrices in the study of general matrices , 1972 .

[38]  Chen Greif,et al.  Preconditioners for the discretized time-harmonic Maxwell equations in mixed form , 2007, Numer. Linear Algebra Appl..

[39]  Cornelis Vuik,et al.  A Novel Multigrid Based Preconditioner For Heterogeneous Helmholtz Problems , 2005, SIAM J. Sci. Comput..

[40]  Thomas A. Manteuffel,et al.  Least-Squares Finite Element Methods and Algebraic Multigrid Solvers for Linear Hyperbolic PDEs , 2004, SIAM J. Sci. Comput..

[41]  W. Hackbusch.  Iterative Solution of Large Sparse Systems of Equations , 1993 .

[42]  Timothy A. Davis,et al.  Algorithm 832: UMFPACK V4.3---an unsymmetric-pattern multifrontal method , 2004, TOMS.

[43]  Leslie Greengard,et al.  A fast algorithm for particle simulations , 1987 .

[44]  Luc Giraud,et al.  Flexible GMRES with Deflated Restarting , 2010, SIAM J. Sci. Comput..

[45]  J. Boyle,et al.  Solvers for large-displacement fluid–structure interaction problems: segregated versus monolithic approaches , 2008 .

[46]  Arnold Reusken,et al.  Robust Parallel Smoothing for Multigrid Via Sparse Approximate Inverses , 2001, SIAM J. Sci. Comput..

[47]  A. Wathen.  Realistic Eigenvalue Bounds for the Galerkin Mass Matrix , 1987 .

[48]  Howard C. Elman,et al.  Algebraic Analysis of the Hierarchical Basis Preconditioner , 1995, SIAM J. Matrix Anal. Appl..

[49]  Barry Smith,et al.  Domain Decomposition Methods for Partial Differential Equations , 1997 .

[50]  John N. Shadid,et al.  Block Preconditioners Based on Approximate Commutators , 2005, SIAM J. Sci. Comput..

[51]  M. Benzi,et al.  INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN FLUIDS Int. J. Numer. Meth. Fluids (2010) Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/fld.2267 Modified augmented Lagrangian preconditioners for the incompressible Navier , 2022 .

[52]  Michele Benzi,et al.  A Sparse Approximate Inverse Preconditioner for Nonsymmetric Linear Systems , 1998, SIAM J. Sci. Comput..

[53]  Kurt Otto,et al.  Analysis of Preconditioners for Hyperbolic Partial Differential Equations , 1996 .

[54]  Axel Klawonn,et al.  Block-Triangular Preconditioners for Saddle Point Problems with a Penalty Term , 1998, SIAM J. Sci. Comput..

[55]  Martin Stoll,et al.  All-at-once preconditioning in PDE-constrained optimization , 2010, Kybernetika.

[56]  R. P. Kendall,et al.  An Approximate Factorization Procedure for Solving Self-Adjoint Elliptic Difference Equations , 1968 .

[57]  Sandro Manservisi,et al.  Numerical Analysis of Vanka-Type Solvers for Steady Stokes and Navier-Stokes Flows , 2006, SIAM J. Numer. Anal..

[58]  Maxim A. Olshanskii,et al.  Field-of-Values Convergence Analysis of Augmented Lagrangian Preconditioners for the Linearized Navier-Stokes Problem , 2011, SIAM J. Numer. Anal..

[59]  Lexing Ying,et al.  A Parallel Sweeping Preconditioner for Heterogeneous 3D Helmholtz Equations , 2012, SIAM J. Sci. Comput..

[60]  Howard C. Elman,et al.  Fast Nonsymmetric Iterations and Preconditioning for Navier-Stokes Equations , 1996, SIAM J. Sci. Comput..

[61]  M. Schultz,et al.  Preconditioning by fast direct methods for nonself-adjoint nonseparable elliptic equations , 1986 .

[62]  M. Arioli,et al.  A stopping criterion for the conjugate gradient algorithm in a finite element method framework , 2000, Numerische Mathematik.

[63]  J. Pasciak,et al.  Parallel multilevel preconditioners , 1990 .

[64]  H. Elman,et al.  Ordering Effects on Relaxation Methods Applied to The Discrete One-Dimensional Convection-Diffusion , 1992 .

[65]  Lexing Ying,et al.  A sweeping preconditioner for time-harmonic Maxwell's equations with finite elements , 2012, J. Comput. Phys..

[66]  Shang-Hua Teng,et al.  Nearly-Linear Time Algorithms for Preconditioning and Solving Symmetric, Diagonally Dominant Linear Systems , 2006, SIAM J. Matrix Anal. Appl..

[67]  Andrea Toselli,et al.  Domain decomposition methods : algorithms and theory , 2005 .

[68]  Andrew J. Wathen,et al.  Approximate Factorization Constraint Preconditioners for Saddle-Point Matrices , 2005, SIAM J. Sci. Comput..

[69]  Martin J. Gander,et al.  Why it is Difficult to Solve Helmholtz Problems with Classical Iterative Methods , 2012 .

[70]  Valeria Simoncini,et al.  Recent computational developments in Krylov subspace methods for linear systems , 2007, Numer. Linear Algebra Appl..

[71]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[72]  Nicholas I. M. Gould,et al.  On the Solution of Equality Constrained Quadratic Programming Problems Arising in Optimization , 2001, SIAM J. Sci. Comput..

[73]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[74]  Nicholas I. M. Gould,et al.  Spectral Analysis of Saddle Point Matrices with Indefinite Leading Blocks , 2009, SIAM J. Matrix Anal. Appl..

[75]  Danny C. Sorensen,et al.  Deflation Techniques for an Implicitly Restarted Arnoldi Iteration , 1996, SIAM J. Matrix Anal. Appl..

[76]  Achi Brandt,et al.  Lean Algebraic Multigrid (LAMG): Fast Graph Laplacian Linear Solver , 2011, SIAM J. Sci. Comput..

[77]  Michele Benzi,et al.  Robust Approximate Inverse Preconditioning for the Conjugate Gradient Method , 2000, SIAM J. Sci. Comput..

[78]  A. Ramage.  A multigrid preconditioner for stabilised discretisations of advection-diffusion problems , 1999 .

[79]  J. Meijerink,et al.  An iterative solution method for linear systems of which the coefficient matrix is a symmetric -matrix , 1977 .

[80]  D. Braess,et al.  On the numerical solution of the biharmonic equation and the role of squaring matrices , 1986 .

[81]  Laura Schweitzer.  Matrix Preconditioning Techniques And Applications , 2016 .

[82]  A. Wathen,et al.  Chebyshev semi-iteration in preconditioning for problems including the mass matrix. , 2008 .

[83]  Wolfgang Hackbusch,et al.  A Sparse Matrix Arithmetic Based on H-Matrices. Part I: Introduction to H-Matrices , 1999, Computing.

[84]  C. Loan.  Computational Frameworks for the Fast Fourier Transform , 1992 .

[85]  Marcus J. Grote,et al.  Parallel Preconditioning with Sparse Approximate Inverses , 1997, SIAM J. Sci. Comput..

[86]  Michele Benzi,et al.  On the eigenvalues of a class of saddle point matrices , 2006, Numerische Mathematik.

[87]  Michael F. Drummond,et al.  A Final Comment , 1985 .

[88]  L. Trefethen.  Spectra and pseudospectra , 2005 .

[89]  Andrew J. Wathen,et al.  On the choice of preconditioner for minimum residual methods for non-Hermitian matrices , 2013, J. Comput. Appl. Math..

[90]  Ronald B. Morgan,et al.  GMRES Convergence for Perturbed Coefficient Matrices, with Application to Approximate Deflation Preconditioning , 2013, SIAM J. Matrix Anal. Appl..

[91]  T. Manteuffel,et al.  Necessary and Sufficient Conditions for the Existence of a Conjugate Gradient Method , 1984 .

[92]  P. Vassilevski.  Multilevel Block Factorization Preconditioners: Matrix-based Analysis and Algorithms for Solving Finite Element Equations , 2008 .

[93]  J. Tukey,et al.  An algorithm for the machine calculation of complex Fourier series , 1965 .

[94]  O. Widlund.  A Lanczos Method for a Class of Nonsymmetric Systems of Linear Equations , 1978 .

[95]  Michael B. Giles,et al.  Preconditioned iterative solution of the 2D Helmholtz equation , 2002 .

[96]  Jinchao Xu,et al.  Iterative Methods by Space Decomposition and Subspace Correction , 1992, SIAM Rev..

[97]  O. Widlund.  Domain Decomposition Algorithms , 1993 .

[98]  Dianne P. O'Leary,et al.  A Multigrid Method Enhanced by Krylov Subspace Iteration for Discrete Helmholtz Equations , 2001, SIAM J. Sci. Comput..

[99]  E. Haber,et al.  Preconditioned all-at-once methods for large, sparse parameter estimation problems , 2001 .

[100]  B. Engquist,et al.  Sweeping preconditioner for the Helmholtz equation: Hierarchical matrix representation , 2010, 1007.4290.

[101]  R. Verfürth.  A combined conjugate gradient - multi-grid algorithm for the numerical solution of the Stokes problem , 1984 .

[102]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[103]  L. Kolotilina,et al.  Factorized Sparse Approximate Inverse Preconditionings I. Theory , 1993, SIAM J. Matrix Anal. Appl..

[104]  Nicholas I. M. Gould,et al.  Implicit-Factorization Preconditioning and Iterative Solvers for Regularized Saddle-Point Systems , 2006, SIAM J. Matrix Anal. Appl..

[105]  Tarek P. Mathew,et al.  Domain Decomposition Methods for the Numerical Solution of Partial Differential Equations , 2008, Lecture Notes in Computational Science and Engineering.

[106]  Volker John,et al.  International Journal for Numerical Methods in Fluids Numerical Performance of Smoothers in Coupled Multigrid Methods for the Parallel Solution of the Incompressible Navier – Stokes Equations , 2022 .

[107]  Maria Elizabeth G. Ong,et al.  Hierarchical Basis Preconditioners in Three Dimensions , 1997, SIAM J. Sci. Comput..

[108]  G. Strang.  A proposal for toeplitz matrix calculations , 1986 .

[109]  A. Wathen,et al.  Iterative Methods for Toeplitz Systems , 2005 .

[110]  Yousef Saad,et al.  A Flexible Inner-Outer Preconditioned GMRES Algorithm , 1993, SIAM J. Sci. Comput..

[111]  Xiaoye S. Li,et al.  An overview of SuperLU: Algorithms, implementation, and user interface , 2003, TOMS.

[112]  Martin J. Gander,et al.  Optimized Schwarz Methods without Overlap for the Helmholtz Equation , 2002, SIAM J. Sci. Comput..

[113]  Andrew J. Wathen,et al.  A Preconditioner for the Steady-State Navier-Stokes Equations , 2002, SIAM J. Sci. Comput..

[114]  Andrew J. Wathen,et al.  Sparse Approximate Inverses and Target Matrices , 2005, SIAM J. Sci. Comput..

[115]  Michele Benzi,et al.  A Sparse Approximate Inverse Preconditioner for the Conjugate Gradient Method , 1996, SIAM J. Sci. Comput..

[116]  Michael B. Giles,et al.  Multilevel Monte Carlo Path Simulation , 2008, Oper. Res..

[117]  Zhen Wang,et al.  A Relaxed Dimensional Factorization preconditioner for the incompressible Navier-Stokes equations , 2011, J. Comput. Phys..

[118]  J. Scott,et al.  HSL MI 20 : An efficient AMG preconditioner for finite element problems in 3 D , 2010 .

[119]  Mario Bebendorf,et al.  Hierarchical Matrices: A Means to Efficiently Solve Elliptic Boundary Value Problems , 2008 .

[120]  Robert Beauwens,et al.  Preconditioning of discrete Helmholtz operators perturbed by a diagonal complex matrix , 2000 .

[121]  R. Freund,et al.  On a class of Chebyshev approximation problems which arise in connection with a conjugate gradient type method , 1986 .

[122]  S. SIAMJ.,et al.  AGGREGATION-BASED ALGEBRAIC MULTIGRID FOR CONVECTION-DIFFUSION EQUATIONS∗ , 2012 .

[123]  V. E. Henson,et al.  BoomerAMG: a parallel algebraic multigrid solver and preconditioner , 2002 .

[124]  N. Bakhvalov.  On the convergence of a relaxation method with natural constraints on the elliptic operator , 1966 .

[125]  John N. Shadid,et al.  A Block Preconditioner for an Exact Penalty Formulation for Stationary MHD , 2014, SIAM J. Sci. Comput..

[126]  Howard C. Elman,et al.  Algorithm 866: IFISS, a Matlab toolbox for modelling incompressible flow , 2007, TOMS.

[127]  Patrick H. Worley.  Limits on Parallelism in the Numerical Solution of Linear Partial Differential Equations , 1991, SIAM J. Sci. Comput..

[128]  Andrew J. Wathen,et al.  Stopping criteria for iterations in finite element methods , 2005, Numerische Mathematik.

[129]  Maxim A. Olshanskii,et al.  Iterative Methods for Linear Systems - Theory and Applications , 2014 .

[130]  Nicholas I. M. Gould,et al.  Constraint Preconditioning for Indefinite Linear Systems , 2000, SIAM J. Matrix Anal. Appl..

[131]  Z. Strakos,et al.  Krylov Subspace Methods: Principles and Analysis , 2012 .

[132]  Ian G. Graham,et al.  Unstructured Additive Schwarz-Conjugate Gradient Method for Elliptic Problems with Highly Discontinuous Coefficients , 1999, SIAM J. Sci. Comput..

[133]  A. Wathen,et al.  Fast iterative solution of stabilised Stokes systems part II: using general block preconditioners , 1994 .

[134]  Anne Greenbaum,et al.  Iterative methods for solving linear systems , 1997, Frontiers in applied mathematics.

[135]  Gene H. Golub,et al.  Hermitian and Skew-Hermitian Splitting Methods for Non-Hermitian Positive Definite Linear Systems , 2002, SIAM J. Matrix Anal. Appl..

[136]  R. Freund,et al.  A new Krylov-subspace method for symmetric indefinite linear systems , 1994 .

[137]  Martin B. van Gijzen,et al.  IDR(s): A Family of Simple and Fast Algorithms for Solving Large Nonsymmetric Systems of Linear Equations , 2008, SIAM J. Sci. Comput..

[138]  D. O’Leary.  Yet another polynomial preconditioner for the conjugate gradient algorithm , 1991 .

[139]  Gene H. Golub,et al.  A Note on Preconditioning for Indefinite Linear Systems , 1999, SIAM J. Sci. Comput..

[140]  Wolfgang Hackbusch,et al.  Construction and Arithmetics of H-Matrices , 2003, Computing.

[141]  Yousef Saad,et al.  ILUM: A Multi-Elimination ILU Preconditioner for General Sparse Matrices , 1996, SIAM J. Sci. Comput..

[142]  Michael Wathen.  Iterative solution of a mixed finite element discretisation of an incompressible magnetohydrodynamics problem , 2014 .

[143]  R. P. Fedorenko.  The speed of convergence of one iterative process , 1964 .

[144]  Achi Brandt,et al.  Multigrid method for nearly singular and slightly indefinite problems , 1986 .

[145]  J. Pasciak,et al.  A preconditioning technique for indefinite systems resulting from mixed approximations of elliptic problems , 1988 .

[146]  Cornelis Vuik,et al.  On the Construction of Deflation-Based Preconditioners , 2001, SIAM J. Sci. Comput..

[147]  A. Wathen.  Preconditioning and convergence in the right norm , 2007 .

[148]  M. Saunders,et al.  Solution of Sparse Indefinite Systems of Linear Equations , 1975 .

[149]  Lexing Ying,et al.  Sweeping Preconditioner for the Helmholtz Equation: Moving Perfectly Matched Layers , 2010, Multiscale Model. Simul..

[150]  Zdenek Strakos,et al.  Preconditioning and the Conjugate Gradient Method in the Context of Solving PDEs , 2014, SIAM spotlights.

[151]  M. B. Van Gijzen,et al.  Comparison of the deflated preconditioned conjugate gradient method and algebraic multigrid for composite materials , 2012 .

[152]  Daniel B. Szyld,et al.  An introduction to iterative Toeplitz solvers , 2009, Math. Comput..

[153]  Gene H. Golub,et al.  Numerical solution of saddle point problems , 2005, Acta Numerica.

[154]  K. Burrage,et al.  Restarted GMRES preconditioned by deflation , 1996 .