Convergence Analysis of Krylov Subspace Iterations with Methods from Potential Theory

Krylov subspace iterations are among the best-known and most widely used numerical methods for solving linear systems of equations and for computing eigenvalues of large matrices. These methods are polynomial methods whose convergence behavior is related to the behavior of polynomials on the spectrum of the matrix. This leads to an extremal problem in polynomial approximation theory: How small can a monic polynomial of a given degree be on the spectrum? This survey gives an introduction to a recently developed technique to analyze this extremal problem in the case of symmetric matrices. It is based on global information on the spectrum in the sense that the eigenvalues are assumed to be distributed according to a certain measure. Then, depending on the number of iterations, the Lanczos method for the calculation of eigenvalues finds those eigenvalues that lie in a certain region, which is characterized by means of a constrained equilibrium problem from potential theory. The same constrained equilibrium problem also describes the superlinear convergence of conjugate gradients and other iterative methods for solving linear systems.

[1]  David S. Watkins,et al.  Some Perspectives on the Eigenvalue Problem , 1993, SIAM Rev..

[2]  H. V. D. Vorst,et al.  The rate of convergence of Conjugate Gradients , 1986 .

[3]  Hrushikesh Narhar Mhaskar,et al.  Extremal problems for polynomials with exponential weights , 1984 .

[4]  Percy Deift.  Four Lectures on Random Matrix Theory , 2003 .

[5]  Steven B. Damelin,et al.  The support of the equilibrium measure in the presence of a monomial external field on [-1,1] , 1999 .

[6]  Marc Van Barel,et al.  Convergence of the Isometric Arnoldi Process , 2005, SIAM J. Matrix Anal. Appl..

[7]  E. Rakhmanov,et al.  Equilibrium measure and the distribution of zeros of the extremal polynomials of a discrete variable , 1996 .

[8]  G. Golub,et al.  Eigenvalue computation in the 20th century , 2000 .

[9]  Hrushikesh Narhar Mhaskar,et al.  Weighted analogues of capacity, transfinite diameter, and Chebyshev constant , 1992 .

[10]  I. Goldsheid,et al.  Eigenvalue curves of asymmetric tridiagonal random matrices , 2000, math-ph/0011003.

[11]  N. S. Landkof.  Foundations of Modern Potential Theory , 1972 .

[12]  K. Johansson.  Discrete orthogonal polynomial ensembles and the Plancherel measure. , 1999, math/9906120.

[13]  O. Nevanlinna.  Convergence of Iterations for Linear Equations , 1993 .

[14]  Jörg Liesen,et al.  The Worst-Case GMRES for Normal Matrices , 2004 .

[15]  Thomas A. Manteuffel,et al.  Minimal Residual Method Stronger than Polynomial Preconditioning , 1996, SIAM J. Matrix Anal. Appl..

[16]  Y. Saad.  On the Rates of Convergence of the Lanczos and the Block-Lanczos Methods , 1980 .

[17]  Arno B. J. Kuijlaars,et al.  On The Sharpness of an Asymptotic Error Estimate for Conjugate Gradients , 2001 .

[18]  T. Guhr,et al.  RANDOM-MATRIX THEORIES IN QUANTUM PHYSICS : COMMON CONCEPTS , 1997, cond-mat/9707301.

[19]  Stephen J. Gardiner,et al.  Classical Potential Theory , 2000 .

[20]  Kim-Chuan Toh,et al.  GMRES vs. Ideal GMRES , 1997, SIAM J. Matrix Anal. Appl..

[21]  Vilmos Totik,et al.  Weighted Approximation with Varying Weight , 1994 .

[22]  Richard Barrett,et al.  Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods , 1994, Other Titles in Applied Mathematics.

[23]  Edward B. Saff,et al.  Jentzsch-Szegö Type Theorems for the Zeros of Best Approximants , 1988 .

[24]  B. Fischer.  Polynomial Based Iteration Methods for Symmetric Linear Systems , 1996 .

[25]  Arno B. J. Kuijlaars,et al.  Zero distributions for discrete orthogonal polynomials , 1998 .

[26]  F. Hiai,et al.  The semicircle law, free random variables, and entropy , 2006 .

[27]  Vilmos Totik,et al.  General Orthogonal Polynomials , 1992 .

[28]  D. Calvetti,et al.  AN IMPLICITLY RESTARTED LANCZOS METHOD FOR LARGE SYMMETRIC EIGENVALUE PROBLEMS , 1994 .

[29]  M. Barel,et al.  A numerical solution of the constrained energy problem , 2006 .

[30]  STEFANO SERRA CAPIZZANO,et al.  Locally X Matrices, Spectral Distributions, Preconditioning, and Applications , 2000, SIAM J. Matrix Anal. Appl..

[31]  Bernhard Beckermann,et al.  A note on the convergence of Ritz values for sequences of matrices , 2000 .

[32]  Tobin A. Driscoll,et al.  From Potential Theory to Matrix Iterations in Six Steps , 1998, SIAM Rev..

[33]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[34]  S. Capizzano.  Spectral behavior of matrix sequences and discretized boundary value problems , 2001 .

[35]  H. V. D. Vorst,et al.  The convergence behavior of ritz values in the presence of close eigenvalues , 1987 .

[36]  Boris A. Khoruzhenko,et al.  Eigenvalue Curves of Asymmetric Tridiagonal Matrices , 2000 .

[37]  Edward B. Saff,et al.  Constrained energy problems with applications to orthogonal polynomials of a discrete variable , 1997 .

[38]  Arno B. J. Kuijlaars,et al.  Superlinear CG convergence for special right-hand sides , 2002 .

[39]  P. Deift.  Orthogonal Polynomials and Random Matrices: A Riemann-Hilbert Approach , 2000 .

[40]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[41]  Arno B. J. Kuijlaars,et al.  Extremal Polynomials on Discrete Sets , 1999 .

[42]  R. Langer.  Interpolation and Approximation by Rational Functions in the Complex Domain , 1937 .

[43]  M. Stephanov,et al.  Random Matrices , 2005, hep-ph/0509286.

[44]  E. Rakhmanov,et al.  ON ASYMPTOTIC PROPERTIES OF POLYNOMIALS ORTHOGONAL ON THE REAL AXIS , 1984 .

[45]  P. D. Miller,et al.  Uniform Asymptotics for Polynomials Orthogonal With Respect to a General Class of Discrete Weights and Universality Results for Associated Ensembles , 2002 .

[46]  C. Tracy,et al.  Introduction to Random Matrices , 1992, hep-th/9210073.

[47]  Edward B. Saff,et al.  A Problem in Potential Theory and Zero Asymptotics of Krawtchouk Polynomials , 2000 .

[48]  Arno B. J. Kuijlaars,et al.  Superlinear Convergence of Conjugate Gradients , 2001, SIAM J. Numer. Anal..

[49]  Anne Greenbaum,et al.  Max-Min Properties of Matrix Factor Norms , 1994, SIAM J. Sci. Comput..

[50]  S. Serra Capizzano,et al.  Generalized locally Toeplitz sequences: spectral analysis and applications to discretized partial differential equations , 2003 .

[51]  Hrushikesh Narhar Mhaskar,et al.  Where does the sup norm of a weighted polynomial live? , 1985 .

[52]  Arno B. J. Kuijlaars,et al.  Which Eigenvalues Are Found by the Lanczos Method? , 2000, SIAM J. Matrix Anal. Appl..

[53]  S. Kaniel.  Estimates for Some Computational Techniques - in Linear Algebra , 1966 .

[54]  Antonia Maria Tulino,et al.  Random Matrix Theory and Wireless Communications , 2004, Found. Trends Commun. Inf. Theory.

[55]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[56]  E. Rakhmanov,et al.  EQUILIBRIUM MEASURE AND THE DISTRIBUTION OF ZEROS OF EXTREMAL POLYNOMIALS , 1986 .

[57]  John Rossi,et al.  Convergence of Restarted Krylov Subspaces to Invariant Subspaces , 2004, SIAM J. Matrix Anal. Appl..

[58]  A. Sluis,et al.  Further results on the convergence behavior of conjugate-gradients and Ritz values , 1996 .

[59]  Thomas Ransford,et al.  Potential Theory in the Complex Plane: Bibliography , 1995 .

[60]  G. Pólya,et al.  Über den transfiniten Durchmesser (Kapazitätskonstante) von ebenen und räumlichen Punktmengen. , 1931 .

[61]  O. Axelsson.  Iterative solution methods , 1995 .

[62]  Percy Deift,et al.  A continuum limit of the Toda lattice , 1998 .

[63]  P. J. Forrester,et al.  Developments in random matrix theory , 2003, cond-mat/0303207.

[64]  E. Saff,et al.  Logarithmic Potentials with External Fields , 1997 .

[65]  W. Van Assche,et al.  Asymptotics of discrete orthogonal polynomials and the continuum limit of the Toda lattice , 2001 .

[66]  D. Sorensen.  Numerical methods for large eigenvalue problems , 2002, Acta Numerica.

[67]  Anne Greenbaum,et al.  Iterative methods for solving linear systems , 1997, Frontiers in applied mathematics.

[68]  Otto Frostman.  Potentiel d'équilibre et capacité des ensembles : Avec quelques applications a la théorie des fonctions , 1935 .

[69]  M. Brelot.  Classical potential theory and its probabilistic counterpart , 1986 .

[70]  Craig A. Tracy,et al.  Universality of the distribution functions of random matrix theory , 1999 .

[71]  K. Johansson.  Non-intersecting paths, random tilings and random matrices , 2000, math/0011250.

[72]  Arno B. J. Kuijlaars,et al.  On the finite-gap ansatz in the continuum limit of the Toda lattice , 2000 .

[73]  L. Trefethen,et al.  Spectra, pseudospectra, and localization for random bidiagonal matrices , 2000, cond-mat/0003514.

[74]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[75]  I.Ya. Goldsheid,et al.  Regular Spacings of Complex Eigenvalues in the One-Dimensional Non-Hermitian Anderson Model , 2003 .

[76]  Lloyd N. Trefethen,et al.  GMRES/CR and Arnoldi/Lanczos as Matrix Approximation Problems , 2018, SIAM J. Sci. Comput..

[77]  E. Rakhmanov,et al.  Families of equilibrium measures in an external field on the real axis , 1999 .

[78]  Gene H. Golub,et al.  Matrix computations , 1983 .