Block Krylov Subspace Methods for Functions of Matrices II: Modified Block FOM

A variety of block Krylov subspace methods have been successfully developed for linear systems and matrix equations. The application of block Krylov methods to compute matrix functions is, however, less established, despite the growing prevalence of matrix functions in scientific computing. Of particular importance is the evaluation of a matrix function on not just one but multiple vectors. The main contribution of this paper is a class of efficient block Krylov subspace methods tailored precisely to this task. With the full orthogonalization method (FOM) for linear systems forming the backbone of our theory, the resulting methods are referred to as B(FOM)2: block FOM for functions of matrices. Many other important results are obtained in the process of developing these new methods. Matrix-valued inner products are used to construct a general framework for block Krylov subspaces that encompasses already established results in the literature. Convergence bounds for B(FOM)2 are proven for Stieltjes functions applied to a class of matrices which are self-adjoint and positive definite with respect to the matrix-valued inner product. A detailed algorithm for B(FOM)2 with restarts is developed, whose efficiency is based on a recursive expression for the error, which is also used to update the solution. Numerical experiments demonstrate the power and versatility of this new class of methods for a variety of matrix-valued inner products, functions, and matrices.

[1]  J. Dennis,et al.  On the Matrix Polynomial, Lambda-Matrix and Block Eigenvalue Problems , 1971 .

[2]  Henry C. Thacher,et al.  Applied and Computational Complex Analysis. , 1988 .

[3]  Joseph F. Traub,et al.  The Algebraic Theory of Matrix Polynomials , 1976 .

[4]  Kenneth G. Wilson,et al.  Quantum Chromodynamics on a Lattice , 1977 .

[5]  Joseph F. Traub,et al.  Algorithms for solvents of matrix polynomials , 1978 .

[6]  D. O’Leary.  The block conjugate gradient algorithm and related methods , 1980 .

[7]  S. Eisenstat,et al.  Variational Iterative Methods for Nonsymmetric Systems of Linear Equations , 1983 .

[8]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[9]  Y. Saad,et al.  On the Lánczos method for solving symmetric linear systems with several right-hand sides , 1987 .

[10]  L. Knizhnerman,et al.  Two polynomial methods of calculating functions of symmetric matrices , 1991 .

[11]  Y. Saad.  Analysis of some Krylov subspace approximations to the matrix exponential operator , 1992 .

[12]  Andreas Frommer,et al.  MANY MASSES ON ONE STROKE: ECONOMIC COMPUTATION OF QUARK PROPAGATORS , 1995 .

[13]  Andy A. Nikishin,et al.  Variable Block CG Algorithms for Solving Large Sparse Symmetric Positive Definite Linear Systems on Parallel Computers, I: General Iterative Scheme , 1995, SIAM J. Matrix Anal. Appl..

[14]  Efstratios Gallopoulos,et al.  An Iterative Method for Nonsymmetric Systems with Multiple Right-Hand Sides , 1995, SIAM J. Sci. Comput..

[15]  Henk A. van der Vorst,et al.  Approximate solutions and eigenvalue bounds from Krylov subspaces , 1995, Numer. Linear Algebra Appl..

[16]  Valeria Simoncini,et al.  RITZ AND PSEUDO-RITZ VALUES USING MATRIX POLYNOMIALS , 1996 .

[17]  V. Simoncini,et al.  Convergence properties of block GMRES and matrix polynomials , 1996 .

[18]  C. G. Broyden.  A breakdown of the block CG method , 1996 .

[19]  G. Starke.  Field-of-values analysis of preconditioned iterative methods for nonsymmetric elliptic problems , 1997 .

[20]  Herbert Neuberger.  Exactly massless quarks on the lattice , 1998 .

[21]  Andreas Frommer,et al.  Restarted GMRES for Shifted Linear Systems , 1998, SIAM J. Sci. Comput..

[22]  H. Sadok,et al.  Global FOM and GMRES algorithms for matrix equations , 1999 .

[23]  Jean-François Richard,et al.  Methods of Numerical Integration , 2000 .

[24]  Roland W. Freund,et al.  A Lanczos-type method for multiple starting vectors , 2000, Math. Comput..

[25]  A. Dubrulle,et al.  Retooling the method of block conjugate gradients. , 2001 .

[26]  Iain S. Duff,et al.  An overview of the sparse basic linear algebra subprograms: The new standard from the BLAS technical forum , 2002, TOMS.

[27]  Peter Lancaster,et al.  Lambda-matrices and vibrating systems , 2002 .

[28]  H. V. D. Vorst,et al.  Numerical methods for the QCDd overlap operator. I. Sign-function and error bounds , 2002, hep-lat/0202025.

[29]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[30]  M. Gutknecht,et al.  Block Krylov methods for Hermitian linear systems , 2004 .

[31]  Khalide Jbilou,et al.  The block Lanczos method for linear systems with multiple right-hand sides , 2004 .

[32]  Philip I. Davies,et al.  Computing f (A)b for matrix functions f , 2005 .

[33]  M. Gutknecht.  BLOCK KRYLOV SPACE METHODS FOR LINEAR SYSTEMS WITH MULTIPLE RIGHT-HAND SIDES : AN , 2005 .

[34]  R. Morgan.  Restarted block-GMRES with deflation of eigenvalues , 2005 .

[35]  M. Heyouni,et al.  Matrix Krylov subspace methods for linear systems with multiple right-hand sides , 2005, Numerical Algorithms.

[36]  L. Trefethen,et al.  Talbot quadratures and rational approximations , 2006 .

[37]  H. Sadok,et al.  Convergence properties of some block Krylov subspace methods for multiple linear systems , 2006 .

[38]  J. A. C. Weideman,et al.  Optimizing Talbot's Contours for the Inversion of the Laplace Transform , 2006, SIAM J. Numer. Anal..

[39]  VALERIA SIMONCINI,et al.  MATRIX FUNCTIONS , 2006 .

[40]  V. Simoncini,et al.  Preserving geometric properties of the exponential matrix by block Krylov subspace methods , 2006 .

[41]  Eric de Sturler,et al.  Recycling Krylov Subspaces for Sequences of Linear Systems , 2006, SIAM J. Sci. Comput..

[42]  Oliver G. Ernst,et al.  A Restarted Krylov Subspace Method for the Evaluation of Matrix Functions , 2006, SIAM J. Numer. Anal..

[43]  Elizabeth R. Jessup,et al.  On Improving Linear Solver Performance: A Block Variant of GMRES , 2005, SIAM J. Sci. Comput..

[44]  N. Higham,et al.  Computing A, log(A) and Related Matrix Functions by Contour Integrals , 2007 .

[45]  Valeria Simoncini,et al.  Recent computational developments in Krylov subspace methods for linear systems , 2007, Numer. Linear Algebra Appl..

[46]  Lloyd N. Trefethen,et al.  Parabolic and hyperbolic contours for computing the Bromwich integral , 2007, Math. Comput..

[47]  Barry Simon,et al.  The Analytic Theory of Matrix Orthogonal Polynomials , 2007, 0711.2703.

[48]  Baojiang Zhong,et al.  Simpler block GMRES for nonsymmetric systems with multiple right-hand sides. , 2008 .

[49]  Nicholas J. Higham,et al.  Functions of matrices - theory and computation , 2008 .

[50]  M. Eiermann,et al.  Implementation of a restarted Krylov subspace method for the evaluation of matrix functions , 2008 .

[51]  N. Higham.  Functions Of Matrices , 2008 .

[52]  Lothar Reichel,et al.  Error Estimates and Evaluation of Matrix Functions via the Faber Transform , 2009, SIAM J. Numer. Anal..

[53]  H. Sadok,et al.  ALGEBRAIC PROPERTIES OF THE BLOCK GMRES AND BLOCK ARNOLDI METHODS , 2009 .

[54]  M. Gutknecht,et al.  The block grade of a block Krylov space , 2009 .

[55]  Klaus Schilling,et al.  Numerical methods for the QCD overlap operator IV: Hybrid Monte Carlo , 2009, Comput. Phys. Commun..

[56]  Valeria Simoncini,et al.  A new investigation of the extended Krylov subspace method for matrix function evaluations , 2009, Numer. Linear Algebra Appl..

[57]  Mark Hoemmen,et al.  Communication-avoiding Krylov subspace methods , 2010 .

[58]  M. Hochbruck,et al.  Exponential integrators , 2010, Acta Numerica.

[59]  I. Turner,et al.  A restarted Lanczos approximation to functions of a symmetric matrix , 2010 .

[60]  Awad H. Al-Mohy,et al.  Computing the Action of the Matrix Exponential, with an Application to Exponential Integrators , 2011, SIAM J. Sci. Comput..

[61]  Jing Zhao,et al.  A new family of global methods for linear systems with multiple right-hand sides , 2011, J. Comput. Appl. Math..

[62]  Timothy A. Davis,et al.  The university of Florida sparse matrix collection , 2011, TOMS.

[63]  Davod Khojasteh Salkuyeh,et al.  On the global Krylov subspace methods for solving general coupled matrix equations , 2011, Comput. Math. Appl..

[64]  Z. Strakos,et al.  Krylov Subspace Methods: Principles and Analysis , 2012 .

[65]  M. Ismail,et al.  Spectral decomposition and matrix-valued orthogonal polynomials , 2012, 1206.4785.

[66]  Gang Wu,et al.  A Preconditioned and Shifted GMRES Algorithm for the PageRank Problem with Multiple Damping Factors , 2012, SIAM J. Sci. Comput..

[67]  Peter K. Kitanidis,et al.  A Flexible Krylov Solver for Shifted Systems with Application to Oscillatory Hydraulic Tomography , 2012, SIAM J. Sci. Comput..

[68]  Kirk M. Soodhalter,et al.  A block MINRES algorithm based on the band Lanczos method , 2013, Numerical Algorithms.

[69]  Dejan Markovic,et al.  A scalable sparse matrix-vector multiplication kernel for energy-efficient sparse-blas on FPGAs , 2014, FPGA.

[70]  André Gaul,et al.  Recycling Krylov subspace methods for sequences of linear systems , 2014 .

[71]  Stefan Güttel,et al.  Convergence of Restarted Krylov Subspace Methods for Stieltjes Functions of Matrices , 2014, SIAM J. Matrix Anal. Appl..

[72]  Kirk M. Soodhalter,et al.  Block Krylov Subspace Recycling for Shifted Systems with Unrelated Right-Hand Sides , 2016, SIAM J. Sci. Comput..

[73]  Laura Grigori,et al.  Enlarged Krylov Subspace Conjugate Gradient Methods for Reducing Communication , 2016, SIAM J. Matrix Anal. Appl..

[74]  Khalide Jbilou,et al.  On some properties of the extended block and global Arnoldi methods with applications to model reduction , 2016, Numerical Algorithms.

[75]  Andreas Frommer,et al.  On short recurrence Krylov type methods for linear systems with many right-hand sides , 2015, J. Comput. Appl. Math..

[76]  Phillipp Kaestner,et al.  Linear And Nonlinear Programming , 2016 .

[77]  Peter Benner,et al.  Frequency-Limited Balanced Truncation with Low-Rank Approximations , 2016, SIAM J. Sci. Comput..

[78]  Michele Benzi,et al.  Approximation of functions of large matrices with Kronecker structure , 2015, Numerische Mathematik.

[79]  Chen Greif,et al.  GMRES with multiple preconditioners , 2017 .

[80]  Daniel B. Szyld,et al.  The Radau-Lanczos Method for Matrix Functions , 2017, SIAM J. Matrix Anal. Appl..

[81]  M. Schweitzer.  Restarting and error estimation in polynomial and extended Krylov subspace methods for the approximation of matrix functions , 2018 .

[82]  Daniel Kressner,et al.  Low-rank updates of matrix functions , 2017, SIAM J. Matrix Anal. Appl..

[83]  Sebastian Birk,et al.  Deflated Shifted Block Krylov Subspace Methods for Hermitian Positive Definite Matrices , 2018 .

[84]  Ting-Zhu Huang,et al.  A block GMRES method with deflated restarting for solving linear systems with multiple shifts and multiple right‐hand sides , 2018, Numer. Linear Algebra Appl..

[85]  Matrices , 2019, Numerical C.