Projected Krylov Methods for Saddle-Point Systems

Projected Krylov methods are full-space formulations of Krylov methods that take place in a nullspace. Provided projections into the nullspace can be computed accurately, those methods only require products between an operator and vectors lying in the nullspace. We provide systematic principles for obtaining the projected form of any well-defined Krylov method. Projected Krylov methods are mathematically equivalent to constraint-preconditioned Krylov methods provided the initial guess is well chosen, but require less memory. As a consequence, there are situations where certain known methods such as MINRES and SYMMLQ are well defined in the presence of an indefinite preconditioner.

[1]  Anne Greenbaum,et al.  Iterative methods for solving linear systems , 1997, Frontiers in applied mathematics.

[2]  Michael A. Saunders,et al.  CG Versus MINRES: An Empirical Comparison , 2012 .

[3]  Barry Lee,et al.  Finite elements and fast iterative solvers: with applications in incompressible fluid dynamics , 2006, Math. Comput..

[4]  Nicholas I. M. Gould,et al.  Constraint Preconditioning for Indefinite Linear Systems , 2000, SIAM J. Matrix Anal. Appl..

[5]  Dominique Orban,et al.  Projected Krylov Methods for Unsymmetric Augmented Systems , 2008 .

[6]  Zhi-Hao Cao.  A Note on Constraint Preconditioning for Nonsymmetric Indefinite Matrices , 2002, SIAM J. Matrix Anal. Appl..

[7]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[8]  Henk A. van der Vorst,et al.  Transpose-free formulations of Lanczos-type methods for nonsymmetric linear systems , 2004, Numerical Algorithms.

[9]  Jan Vlcek,et al.  Indefinitely preconditioned inexact Newton method for large sparse equality constrained non-linear programming problems , 1998, Numer. Linear Algebra Appl..

[10]  A. Bruaset.  A survey of preconditioned iterative methods , 1995 .

[11]  Lloyd N. Trefethen,et al.  How Fast are Nonsymmetric Matrix Iterations? , 1992, SIAM J. Matrix Anal. Appl..

[12]  Andrew J. Wathen,et al.  Approximate Factorization Constraint Preconditioners for Saddle-Point Matrices , 2005, SIAM J. Sci. Comput..

[13]  Andrew J. Wathen,et al.  On choice of preconditioner for minimum residual methods for nonsymmetric matrices , 2011 .

[14]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[15]  Valeria Simoncini,et al.  Krylov Subspace Methods for Saddle Point Problems with Indefinite Preconditioning , 2002, SIAM J. Matrix Anal. Appl..

[16]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[17]  Ilaria Perugia,et al.  Linear Algebra Methods in a Mixed Approximation of Magnetostatic Problems , 1999, SIAM J. Sci. Comput..

[18]  R. Fletcher.  Conjugate gradient methods for indefinite systems , 1976 .

[19]  Nicholas I. M. Gould,et al.  Projected Krylov Methods for Saddle-Point Systems , 2013, SIAM J. Matrix Anal. Appl..

[20]  Nicholas I. M. Gould,et al.  A primal-dual trust-region algorithm for non-convex nonlinear programming , 2000, Math. Program..

[21]  Dominique Orban,et al.  The projected Golub-Kahan process for constrained linear least-squares problems , 2014 .

[22]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[23]  Y. Saad.  Krylov subspace methods for solving large unsymmetric linear systems , 1981 .

[24]  Anne Greenbaum,et al.  Any Nonincreasing Convergence Curve is Possible for GMRES , 1996, SIAM J. Matrix Anal. Appl..

[25]  H. Simon,et al.  Two Conjugate-Gradient-Type Methods for Unsymmetric Linear Equations , 1988 .

[26]  W. Arnoldi.  The principle of minimized iterations in the solution of the matrix eigenvalue problem , 1951 .

[27]  Gene H. Golub,et al.  Numerical solution of saddle point problems , 2005, Acta Numerica.

[28]  Nicholas I. M. Gould,et al.  On practical conditions for the existence and uniqueness of solutions to the general equality quadratic programming problem , 1985, Math. Program..

[29]  C. Lanczos.  An iteration method for the solution of the eigenvalue problem of linear differential and integral operators , 1950 .

[30]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[31]  Nicholas I. M. Gould,et al.  On the Solution of Equality Constrained Quadratic Programming Problems Arising in Optimization , 2001, SIAM J. Sci. Comput..

[32]  P. Sonneveld.  CGS, A Fast Lanczos-Type Solver for Nonsymmetric Linear systems , 1989 .

[33]  Michael A. Saunders,et al.  LSMR: An Iterative Algorithm for Sparse Least-Squares Problems , 2010, SIAM J. Sci. Comput..

[34]  R. Chandra.  Conjugate gradient methods for partial differential equations. , 1978 .

[35]  Nicholas I. M. Gould,et al.  Implicit-Factorization Preconditioning and Iterative Solvers for Regularized Saddle-Point Systems , 2006, SIAM J. Matrix Anal. Appl..

[36]  Roland W. Freund,et al.  An Implementation of the Look-Ahead Lanczos Algorithm for Non-Hermitian Matrices , 1993, SIAM J. Sci. Comput..

[37]  Gene H. Golub,et al.  Calculating the singular values and pseudo-inverse of a matrix , 2007, Milestones in Matrix Computation.

[38]  Ilaria Perugia,et al.  Block-diagonal and indefinite symmetric preconditioners for mixed finite element formulations , 2000, Numer. Linear Algebra Appl..

[39]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[40]  M. Saunders,et al.  Solution of Sparse Indefinite Systems of Linear Equations , 1975 .

[41]  Claude Brezinski,et al.  Transpose-free Lanczos-type algorithms for nonsymmetric linear systems , 1998, Numerical Algorithms.