Limited‐memory polynomial methods for large‐scale matrix functions

Matrix functions are a central topic of linear algebra, and problems requiring their numerical approximation appear increasingly often in scientific computing. We review various limited‐memory methods for the approximation of the action of a large‐scale matrix function on a vector. Emphasis is put on polynomial methods, whose memory requirements are known or prescribed a priori. Methods based on explicit polynomial approximation or interpolation, as well as restarted Arnoldi methods, are treated in detail. An overview of existing software is also given, as well as a discussion of challenging open problems.

[1]  C. W. Clenshaw A note on the summation of Chebyshev series , 1955 .

[2]  G. Forsythe Generation and Use of Orthogonal Polynomials for Data-Fitting with a Digital Computer , 1957 .

[3]  Henry C. Thacher,et al.  Applied and Computational Complex Analysis. , 1988 .

[4]  H. Tal-Ezer,et al.  An accurate and efficient scheme for propagating the time dependent Schrödinger equation , 1984 .

[5]  I. Duff,et al.  Direct Methods for Sparse Matrices , 1987 .

[6]  L. Reichel Newton interpolation at Leja points , 1990 .

[7]  L. Knizhnerman,et al.  Two polynomial methods of calculating functions of symmetric matrices , 1991 .

[8]  Ronnie Kosloff,et al.  Solution of the time-dependent Liouville-von Neumann equation: dissipative evolution , 1992 .

[9]  Y. Saad Analysis of some Krylov subspace approximations to the matrix exponential operator , 1992 .

[10]  R. Coifman,et al.  The fast multipole method for the wave equation: a pedestrian prescription , 1993, IEEE Antennas and Propagation Magazine.

[11]  Vladimir Druskin,et al.  Krylov subspace approximation of eigenpairs and matrix functions in exact and computer arithmetic , 1995, Numer. Linear Algebra Appl..

[12]  C. Lubich,et al.  On Krylov Subspace Approximations to the Matrix Exponential Operator , 1997 .

[13]  Eter,et al.  Faber and Newton Polynomial Integrators for Open-System Density Matrix Propagation , 1998 .

[14]  Roger B. Sidje,et al.  Expokit: a software package for computing matrix exponentials , 1998, TOMS.

[15]  Anne Greenbaum,et al.  Using Nonorthogonal Lanczos Vectors in the Computation of Matrix Functions , 1998, SIAM J. Sci. Comput..

[16]  Kesheng Wu,et al.  Dynamic Thick Restarting of the Davidson, and the Implicitly Restarted Arnoldi Methods , 1998, SIAM J. Sci. Comput..

[17]  J. Baglama,et al.  FAST LEJA POINTS , 1998 .

[18]  Andreas Frommer,et al.  Restarted GMRES for Shifted Linear Systems , 1998, SIAM J. Sci. Comput..

[19]  Kesheng Wu,et al.  Thick-Restart Lanczos Method for Large Symmetric Eigenvalue Problems , 2000, SIAM J. Matrix Anal. Appl..

[20]  Oliver G. Ernst,et al.  Analysis of acceleration strategies for restarted minimal residual methods , 2000 .

[21]  I. Moret,et al.  THE COMPUTATION OF FUNCTIONS OF MATRICES BY TRUNCATED FABER SERIES , 2001 .

[22]  Arno B. J. Kuijlaars,et al.  Superlinear Convergence of Conjugate Gradients , 2001, SIAM J. Numer. Anal..

[23]  I. Moret,et al.  An interpolatory approximation of the matrix exponential based on Faber polynomials , 2001 .

[24]  Alicja Smoktunowicz,et al.  Backward Stability of Clenshaw's Algorithm , 2002 .

[25]  H. V. D. Vorst,et al.  Numerical methods for the QCDd overlap operator. I. Sign-function and error bounds , 2002, hep-lat/0202025.

[26]  Roberto Barrio,et al.  Rounding error bounds for the Clenshaw and Forsythe algorithms for the evaluation of orthogonal polynomial series , 2002 .

[27]  G. W. Stewart,et al.  A Krylov-Schur Algorithm for Large Eigenproblems , 2001, SIAM J. Matrix Anal. Appl..

[28]  Mark Embree,et al.  The Tortoise and the Hare Restart GMRES , 2003, SIAM Rev..

[29]  V. Simoncini Restarted Full Orthogonalization Method for Shifted Linear Systems , 2003 .

[30]  P. Novati A polynomial method based on Fejèr points for the computation of functions of unsymmetric matrices , 2003 .

[31]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[32]  L. Bergamaschi,et al.  Interpolating discrete advection-diffusion propagators at Leja sequences , 2004 .

[33]  Michiel E. Hochstenbach,et al.  Subspace extraction for matrix functions , 2005 .

[34]  Elizabeth R. Jessup,et al.  A Technique for Accelerating the Convergence of Restarted GMRES , 2005, SIAM J. Matrix Anal. Appl..

[35]  Marlis Hochbruck,et al.  Preconditioning Lanczos Approximations to the Matrix Exponential , 2005, SIAM J. Sci. Comput..

[36]  VALERIA SIMONCINI,et al.  MATRIX FUNCTIONS , 2006 .

[37]  Oliver G. Ernst,et al.  A Restarted Krylov Subspace Method for the Evaluation of Matrix Functions , 2006, SIAM J. Numer. Anal..

[38]  N. Higham,et al.  Computing A, log(A) and Related Matrix Functions by Contour Integrals , 2007 .

[39]  R. Hiptmair,et al.  Boundary Element Methods , 2021, Oberwolfach Reports.

[40]  M. Eiermann,et al.  Implementation of a restarted Krylov subspace method for the evaluation of matrix functions , 2008 .

[41]  M. Hochbruck,et al.  Rational approximation to trigonometric operators , 2008 .

[42]  N. Higham Functions Of Matrices , 2008 .

[43]  Nicholas J. Higham,et al.  Computing AAlpha, log(A), and Related Matrix Functions by Contour Integrals , 2008, SIAM J. Numer. Anal..

[44]  C. Lubich From Quantum to Classical Molecular Dynamics: Reduced Models and Numerical Analysis , 2008 .

[45]  Constantine Bekas,et al.  Computation of Large Invariant Subspaces Using Polynomial Filtered Lanczos Iterations with Applications in Density Functional Theory , 2008, SIAM J. Matrix Anal. Appl..

[46]  Simon Heybrock,et al.  Krylov subspace methods and the sign function: multishifts and deflation in the non-Hermitian case , 2009, 0910.2927.

[47]  Lothar Reichel,et al.  Error Estimates and Evaluation of Matrix Functions via the Faber Transform , 2009, SIAM J. Numer. Anal..

[48]  G. Golub,et al.  Matrices, Moments and Quadrature with Applications , 2009 .

[49]  Awad H. Al-Mohy,et al.  Computing matrix functions , 2010, Acta Numerica.

[50]  M. Hochbruck,et al.  Exponential integrators , 2010, Acta Numerica.

[51]  I. Turner,et al.  A restarted Lanczos approximation to functions of a symmetric matrix , 2010 .

[52]  Awad H. Al-Mohy,et al.  Computing the Action of the Matrix Exponential, with an Application to Exponential Integrators , 2011, SIAM J. Sci. Comput..

[53]  L. Reichel,et al.  Fractional Tikhonov regularization for linear discrete ill-posed problems , 2011 .

[54]  Stefan Güttel,et al.  Automated parameter selection for rational Arnoldi approximation of Markov functions , 2011 .

[55]  Stefan Güttel,et al.  Deflated Restarting for Matrix Functions , 2011, SIAM J. Matrix Anal. Appl..

[56]  Mihai Anitescu,et al.  Computing f(A)b via Least Squares Polynomial Approximations , 2011, SIAM J. Sci. Comput..

[57]  Julien Langou,et al.  Any admissible cycle‐convergence behavior is possible for restarted GMRES at its initial cycles , 2011, Numer. Linear Algebra Appl..

[58]  Nicholas Hale,et al.  An Efficient Implicit FEM Scheme for Fractional-in-Space Reaction-Diffusion Equations , 2012, SIAM J. Sci. Comput..

[59]  Stefan Güttel,et al.  Superlinear convergence of the rational Arnoldi method for the approximation of matrix functions , 2012, Numerische Mathematik.

[60]  Gérard Meurant,et al.  Any Ritz Value Behavior Is Possible for Arnoldi and for GMRES , 2012, SIAM J. Matrix Anal. Appl..

[61]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[62]  Martin J. Gander,et al.  PARAEXP: A Parallel Integrator for Linear Initial-Value Problems , 2013, SIAM J. Sci. Comput..

[63]  Bernhard Beckermann,et al.  Spectral Sets , 2013, 1302.0546.

[64]  S. Güttel Rational Krylov approximation of matrix functions: Numerical methods and optimal pole selection , 2013 .

[65]  A. Ostermann,et al.  Comparison of software for computing the action of the matrix exponential , 2014 .

[66]  Jun-Feng Yin and Guo-Jian Yin Restarted Full Orthogonalization Method with Deflation for Shifted Linear Systems , 2014 .

[67]  Christian Hoelbling Lattice QCD: Concepts, Techniques and Some Results , 2014 .

[68]  Pierre Vandergheynst,et al.  GSPBOX: A toolbox for signal processing on graphs , 2014, ArXiv.

[69]  James Demmel,et al.  Communication lower bounds and optimal algorithms for numerical linear algebra*† , 2014, Acta Numerica.

[70]  Stefan Güttel,et al.  Convergence of Restarted Krylov Subspace Methods for Stieltjes Functions of Matrices , 2014, SIAM J. Matrix Anal. Appl..

[71]  Stefan Güttel,et al.  Efficient and Stable Arnoldi Restarts for Matrix Functions Based on Quadrature , 2014, SIAM J. Matrix Anal. Appl..

[72]  A. Ostermann,et al.  A residual based error estimate for Leja interpolation of matrix functions , 2014 .

[73]  Paolo Novati,et al.  Numerical approximation to the fractional derivative operator , 2014, Numerische Mathematik.

[74]  DOMINIK L. MICHELS,et al.  Exponential integrators for stiff elastodynamic problems , 2014, ACM Trans. Graph..

[75]  Nicholas J. Higham,et al.  A Catalogue of Software for Matrix Functions. Version 1.0 , 2014 .

[76]  G. Meurant,et al.  ON THE ADMISSIBLE CONVERGENCE CURVES FOR RESTARTED GMRES , 2014 .

[77]  W. Hackbusch,et al.  Hierarchical Matrices: Algorithms and Analysis , 2015 .

[78]  P. Vandergheynst,et al.  Accelerated filtering on graphs using Lanczos method , 2015, 1509.04537.

[79]  Yousef Saad,et al.  Approximating Spectral Densities of Large Matrices , 2013, SIAM Rev..

[80]  Laura Grigori,et al.  Enlarged Krylov Subspace Conjugate Gradient Methods for Reducing Communication , 2016, SIAM J. Matrix Anal. Appl..

[81]  VLADIMIR DRUSKIN,et al.  Near-Optimal Perfectly Matched Layers for Indefinite Helmholtz Problems , 2015, SIAM Rev..

[82]  Andreas Frommer,et al.  Error bounds and estimates for Krylov subspace approximations of Stieltjes matrix functions , 2016 .

[83]  Marco Caliari,et al.  The Leja Method Revisited: Backward Error Analysis for the Matrix Exponential , 2015, SIAM J. Sci. Comput..

[84]  Matthias Hein,et al.  Clustering Signed Networks with the Geometric Mean of Laplacians , 2016, NIPS.

[85]  Gerhard Wellein,et al.  High-performance implementation of Chebyshev filter diagonalization for interior eigenvalue computations , 2015, J. Comput. Phys..

[86]  Pascal Frossard,et al.  Learning Heat Diffusion Graphs , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[87]  Awad H. Al-Mohy A New Algorithm for Computing the Actions of Trigonometricand Hyperbolic Matrix Functions , 2017 .

[88]  Daniel B. Szyld,et al.  The Radau-Lanczos Method for Matrix Functions , 2017, SIAM J. Matrix Anal. Appl..

[89]  Ronald B. Morgan,et al.  Weighted Inner Products for GMRES and GMRES-DR , 2017, SIAM J. Sci. Comput..

[90]  Svetozar Margenov,et al.  Parallel solvers for fractional power diffusion problems , 2017, Concurr. Comput. Pract. Exp..

[91]  David Bolin,et al.  Weak convergence of Galerkin approximations for fractional elliptic stochastic PDEs with spatial white noise , 2017, BIT Numerical Mathematics.

[92]  M. Schweitzer Restarting and error estimation in polynomial and extended Krylov subspace methods for the approximation of matrix functions , 2018 .

[93]  Pascal Frossard,et al.  Distributed Signal Processing via Chebyshev Polynomial Approximation , 2011, IEEE Transactions on Signal and Information Processing over Networks.

[94]  Awad H. Al-Mohy A Truncated Taylor Series Algorithm for Computing the Action of Trigonometric and Hyperbolic Matrix Functions , 2018, SIAM J. Sci. Comput..

[95]  Yavor Vutov,et al.  Optimal solvers for linear systems with fractional powers of sparse SPD matrices , 2016, Numer. Linear Algebra Appl..

[96]  D. Szyld,et al.  Block Krylov Subspace Methods for Functions of Matrices II: Modified Block FOM , 2020, SIAM J. Matrix Anal. Appl..

[97]  Gang Wu,et al.  A shifted block FOM algorithm with deflated restarting for matrix exponential computations , 2018 .

[98]  Yoel Shkolnisky,et al.  Matrix Chebyshev expansion and its application to eigenspaces recovery , 2015 .

[99]  Matrices , 2019, Numerical C.

[100]  Anthony P. Austin,et al.  Stable Computation of Generalized Matrix Functions via Polynomial Interpolation , 2019, SIAM J. Matrix Anal. Appl..

[101]  Yousef Saad,et al.  The Eigenvalues Slicing Library (EVSL): Algorithms, Implementation, and Software , 2018, SIAM J. Sci. Comput..

[102]  Daniel Kressner,et al.  A Krylov Subspace Method for the Approximation of Bivariate Matrix Functions , 2018, Structured Matrices in Numerical Linear Algebra.

[103]  Daniel B. Szyld,et al.  Block Krylov Subspace Methods for Functions of Matrices II: Modified Block FOM , 2020, SIAM J. Matrix Anal. Appl..

[104]  Michele Benzi,et al.  Matrix functions in network analysis , 2020, GAMM-Mitteilungen.

[105]  Mike A. Botchev,et al.  ART: adaptive residual-time restarting for Krylov subspace matrix exponential evaluations , 2018, J. Comput. Appl. Math..

[106]  Daniel Kressner,et al.  Compress‐and‐restart block Krylov subspace methods for Sylvester matrix equations , 2020, Numer. Linear Algebra Appl..

[107]  Stefan Güttel,et al.  A comparison of limited-memory Krylov methods for Stieltjes functions of Hermitian matrices , 2020, SIAM J. Matrix Anal. Appl..