GMRES with multiple preconditioners

We propose a variant of GMRES, where multiple (two or more) preconditioners are applied simultaneously, while maintaining minimal residual optimality properties. To accomplish this, a block version of Flexible GMRES is used, but instead of considering blocks associated with multiple right hand sides, we consider a single right-hand side and grow the space by applying each of the preconditioners to all current search directions, minimizing the residual norm over the resulting larger subspace. To alleviate the difficulty of rapidly increasing storage requirements, we present a heuristic limited-memory selective algorithm, and demonstrate the effectiveness of this approach.

[1]  Pierre Gosselet,et al.  A domain decomposition strategy to efficiently solve structures containing repeated patterns , 2009, 1208.6387.

[2]  Yousef Saad,et al.  A Flexible Inner-Outer Preconditioned GMRES Algorithm , 1993, SIAM J. Sci. Comput..

[3]  B. Vital.  Etude de quelques methodes de resolution de problemes lineaires de grande taille sur multiprocesseur , 1990 .

[4]  Robert Bridson,et al.  A Multipreconditioned Conjugate Gradient Algorithm , 2005, SIAM J. Matrix Anal. Appl..

[5]  Henri Calandra,et al.  Flexible Variants of Block Restarted GMRES Methods with Application to Geophysics , 2012, SIAM J. Sci. Comput..

[6]  Peter K. Kitanidis,et al.  Multipreconditioned Gmres for Shifted Systems , 2016, SIAM J. Sci. Comput..

[7]  Stefan Ulbrich,et al.  Optimization with PDE Constraints , 2008, Mathematical modelling.

[8]  L. Luksan,et al.  Indefinitely preconditioned inexact Newton method for large sparse equality constrained non‐linear programming problems , 1998 .

[9]  D. O’Leary,et al.  Multi-Splittings of Matrices and Parallel Solution of Linear Systems , 1985 .

[10]  Andrea Toselli,et al.  Domain decomposition methods : algorithms and theory , 2005 .

[11]  S. Lennart Johnsson,et al.  Alternating direction methods on multiprocessors , 1987 .

[12]  Marcus Sarkis,et al.  Optimal left and right additive Schwarz preconditioning for minimal residual methods with Euclidean and energy norms , 2007 .

[13]  Joachim Schöberl,et al.  Symmetric Indefinite Preconditioners for Saddle Point Problems with Applications to PDE-Constrained Optimization Problems , 2007, SIAM J. Matrix Anal. Appl..

[14]  Andrew J. Wathen,et al.  Optimal Solvers for PDE-Constrained Optimization , 2010, SIAM J. Sci. Comput..

[15]  V. Simoncini,et al.  Convergence properties of block GMRES and matrix polynomials , 1996 .

[16]  Anne Greenbaum,et al.  Any Nonincreasing Convergence Curve is Possible for GMRES , 1996, SIAM J. Matrix Anal. Appl..

[17]  R. S. Chen,et al.  Multipreconditioned GMRES method for electromagnetic wave scattering problems , 2008 .

[18]  Andrew J. Wathen,et al.  A Preconditioner for the Steady-State Navier-Stokes Equations , 2002, SIAM J. Sci. Comput..

[19]  Laura Grigori,et al.  Enlarged Krylov Subspace Conjugate Gradient Methods for Reducing Communication , 2016, SIAM J. Matrix Anal. Appl..

[20]  Martin J. Gander,et al.  Domain Decomposition Methods in Science and Engineering XXI , 2014 .

[21]  D. O’Leary,et al.  A Krylov multisplitting algorithm for solving linear systems of equations , 1993 .

[22]  Jörg Liesen,et al.  A Framework for Deflated and Augmented Krylov Subspace Methods , 2012, SIAM J. Matrix Anal. Appl..

[23]  Y. Notay.  An aggregation-based algebraic multigrid method , 2010 .

[24]  Tony F. Chan,et al.  Analysis of Projection Methods for Solving Linear Systems with Multiple Right-Hand Sides , 1997, SIAM J. Sci. Comput..

[25]  John N. Shadid,et al.  Block Preconditioners Based on Approximate Commutators , 2005, SIAM J. Sci. Comput..

[26]  M. Sadkane,et al.  Exact and inexact breakdowns in the block GMRES method , 2006 .

[27]  H. Elman,et al.  Efficient preconditioning of the linearized NavierStokes equations for incompressible flow , 2001 .

[28]  A. Wathen.  Nonstandard inner products and preconditioned iterative methods , 2011 .

[29]  Martin Stoll,et al.  Combination Preconditioning and the Bramble-Pasciak+ Preconditioner , 2008, SIAM J. Matrix Anal. Appl..

[30]  Jan Vlcek,et al.  Indefinitely preconditioned inexact Newton method for large sparse equality constrained non-linear programming problems , 1998, Numer. Linear Algebra Appl..

[31]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[32]  M. Sadkane.  A block Arnoldi-Chebyshev method for computing the leading eigenpairs of large sparse unsymmetric matrices , 1993 .

[33]  R. Freund.  Model reduction methods based on Krylov subspaces , 2003, Acta Numerica.

[34]  Valeria Simoncini,et al.  On the Occurrence of Superlinear Convergence of Exact and Inexact Krylov Subspace Methods , 2005, SIAM Rev..

[35]  Xiao-Chuan Cai,et al.  A Restricted Additive Schwarz Preconditioner for General Sparse Linear Systems , 1999, SIAM J. Sci. Comput..

[36]  Zhong-Zhi Bai,et al.  Block preconditioners for elliptic PDE-constrained optimization problems , 2011, Computing.

[37]  Nicholas I. M. Gould,et al.  Constraint Preconditioning for Indefinite Linear Systems , 2000, SIAM J. Matrix Anal. Appl..

[38]  D. O’Leary.  The block conjugate gradient algorithm and related methods , 1980 .

[39]  Howard C. Elman,et al.  Finite Elements and Fast Iterative Solvers: with Applications in Incompressible Fluid Dynamics , 2014 .

[40]  Valeria Simoncini,et al.  Recent computational developments in Krylov subspace methods for linear systems , 2007, Numer. Linear Algebra Appl..

[41]  E. Sturler,et al.  Nested Krylov methods based on GCR , 1996 .

[42]  Blanca Ayuso de Dios,et al.  A Combined Preconditioning Strategy for Nonsymmetric Systems , 2012, SIAM J. Sci. Comput..

[43]  Abul Hasan Siddiqi,et al.  Modern Mathematical Models, Methods And Algorithms for Real World Systems , 2007 .

[44]  H. Sadok,et al.  ALGEBRAIC PROPERTIES OF THE BLOCK GMRES AND BLOCK ARNOLDI METHODS , 2009 .

[45]  Axel Ruhe.  Implementation aspects of band Lanczos algorithms for computation of eigenvalues of large sparse sym , 1979 .

[46]  David Young,et al.  Alternating Direction Implicit Methods , 1962, Adv. Comput..

[47]  Jennifer A. Scott,et al.  A parallel direct solver for large sparse highly unsymmetric linear systems , 2004, TOMS.

[48]  M. Benzi.  Preconditioning techniques for large linear systems: a survey , 2002 .

[49]  Barry Lee,et al.  Finite elements and fast iterative solvers: with applications in incompressible fluid dynamics , 2006, Math. Comput..

[50]  H. Elman,et al.  Efficient preconditioning of the linearized Navier-Stokes , 1999 .

[51]  Andrew J. Wathen,et al.  Combination preconditioning of saddle point systems for positive definiteness , 2013, Numer. Linear Algebra Appl..

[52]  Gerard L. G. Sleijpen,et al.  Differences in the Effects of Rounding Errors in Krylov Solvers for Symmetric Indefinite Linear Systems , 2000, SIAM J. Matrix Anal. Appl..

[53]  Chen Greif,et al.  Additive Schwarz with variable weights , 2014 .

[54]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[55]  F. Tröltzsch.  Optimal Control of Partial Differential Equations: Theory, Methods and Applications , 2010 .

[56]  Daniel B. Szyld,et al.  An Algebraic Convergence Theory for Restricted Additive Schwarz Methods Using Weighted Max Norms , 2001, SIAM J. Numer. Anal..

[57]  M. Sadkane.  Block-Arnoldi and Davidson methods for unsymmetric large eigenvalue problems , 1993 .

[58]  Andrew J. Wathen,et al.  A new approximation of the Schur complement in preconditioners for PDE‐constrained optimization , 2012, Numer. Linear Algebra Appl..