Smoothed Analysis for the Conjugate Gradient Algorithm

The purpose of this paper is to establish bounds on the rate of convergence of the conjugate gradient algorithm when the underlying matrix is a random positive definite perturbation of a deterministic positive definite matrix. We estimate all finite moments of a natural halting time when the random perturbation is drawn from the Laguerre unitary ensemble in a critical scaling regime explored in Deift et al. (2016). These estimates are used to analyze the expected iteration count in the framework of smoothed analysis, introduced by Spielman and Teng (2001). The rigorous results are compared with numerical calculations in several cases of interest.

[1]  James Hardy Wilkinson,et al.  Error Analysis of Direct Methods of Matrix Inversion , 1961, JACM.

[2]  P. Deift,et al.  How long does it take to compute the eigenvalues of a random, symmetric matrix? , 2012, 1203.4635.

[3]  Gene H. Golub,et al.  Matrix computations , 1983 .

[4]  D. Spielman,et al.  Smoothed analysis of algorithms: Why the simplex algorithm usually takes polynomial time , 2004 .

[5]  Rene F. Swarttouw,et al.  Orthogonal polynomials , 2020, NIST Handbook of Mathematical Functions.

[6]  P. Deift,et al.  Universality in numerical computations with random data , 2014, Proceedings of the National Academy of Sciences.

[7]  Anne Greenbaum,et al.  Predicting the Behavior of Finite Precision Lanczos and Conjugate Gradient Computations , 2015, SIAM J. Matrix Anal. Appl..

[8]  Z. Strakos,et al.  Krylov Subspace Methods: Principles and Analysis , 2012 .

[9]  A. Edelman.  Eigenvalues and condition numbers of random matrices , 1988 .

[10]  Yann LeCun,et al.  Universality in halting time and its applications in optimization , 2015, ArXiv.

[11]  B. Simon.  Trace ideals and their applications , 1979 .

[12]  P. Deift,et al.  On the condition number of the critically-scaled Laguerre Unitary Ensemble , 2015, 1507.00750.

[13]  P. Deift.  Orthogonal Polynomials and Random Matrices: A Riemann-Hilbert Approach , 2000 .

[14]  Zdenek Strakos,et al.  Krylov Subspace Methods , 2012 .

[15]  P. Forrester.  The spectrum edge of random matrix ensembles , 1993 .

[16]  P. Deift,et al.  Universality for the Toda Algorithm to Compute the Largest Eigenvalue of a Random Matrix , 2016, 1604.07384.

[17]  P. Deift,et al.  Universality for the Toda algorithm to compute the eigenvalues of a random matrix , 2016 .

[18]  Ronald F. Boisvert,et al.  NIST Handbook of Mathematical Functions , 2010 .

[19]  H. Rutishauser.  Theory of Gradient Methods , 1959 .

[20]  A. Greenbaum.  Behavior of slightly perturbed Lanczos and conjugate-gradient recurrences , 1989 .

[21]  Shang-Hua Teng,et al.  Smoothed analysis of algorithms: why the simplex algorithm usually takes polynomial time , 2001, STOC '01.

[22]  Daniel A. Spielman.  The Smoothed Analysis of Algorithms , 2005, FCT.

[23]  V. Marčenko,et al.  DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES , 1967 .

[24]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .