Conditioning of Leverage Scores and Computation by QR Decomposition

The leverage scores of a full-column rank matrix A are the squared row norms of any orthonormal basis for range(A). We show that corresponding leverage scores of two matrices A and A + \Delta A are close in the relative sense, if they have large magnitude and if all principal angles between the column spaces of A and A + \Delta A are small. We also show three classes of bounds that are based on perturbation results of QR decompositions. They demonstrate that relative differences between individual leverage scores strongly depend on the particular type of perturbation \Delta A. The bounds imply that the relative accuracy of an individual leverage score depends on: its magnitude and the two-norm condition of A, if \Delta A is a general perturbation; the two-norm condition number of A, if \Delta A is a perturbation with the same norm-wise row-scaling as A; (to first order) neither condition number nor leverage score magnitude, if \Delta A is a component-wise row-scaled perturbation. Numerical experiments confirm the qualitative and quantitative accuracy of our bounds.

[1]  X.-W. Chang.  On the perturbation of the Q-factor of the QR factorization , 2012, Numer. Linear Algebra Appl..

[2]  Ji-guang Sun.  Perturbation bounds for the Cholesky andQR factorizations , 1991 .

[3]  G. Stewart.  Perturbation Bounds for the $QR$ Factorization of a Matrix , 1977 .

[4]  Michael W. Mahoney.  Randomized Algorithms for Matrices and Data , 2011, Found. Trends Mach. Learn..

[5]  Deng-Yuan Huang,et al.  Influential Observations , 2011, International Encyclopedia of Statistical Science.

[6]  P. Wedin.  On angles between subspaces of a finite dimensional inner product space , 1983 .

[7]  Roy E. Welsch,et al.  Efficient Computing of Regression Diagnostics , 1981 .

[8]  Christos Boutsidis,et al.  An improved approximation algorithm for the column subset selection problem , 2008, SODA.

[9]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[10]  Malik Magdon-Ismail,et al.  Row Sampling for Matrix Algorithms via a Non-Commutative Bernstein Bound , 2010, ArXiv.

[11]  Petros Drineas,et al.  CUR matrix decompositions for improved data analysis , 2009, Proceedings of the National Academy of Sciences.

[12]  S. Muthukrishnan,et al.  Relative-Error CUR Matrix Decompositions , 2007, SIAM J. Matrix Anal. Appl..

[13]  S. Muthukrishnan,et al.  Sampling algorithms for l2 regression and applications , 2006, SODA '06.

[14]  Ameet Talwalkar,et al.  Matrix Coherence and the Nystrom Method , 2010, UAI.

[15]  V. N. Bogaevski,et al.  Matrix Perturbation Theory , 1991 .

[16]  R. Welsch,et al.  The Hat Matrix in Regression and ANOVA , 1978 .

[17]  David P. Woodruff,et al.  Fast approximation of matrix coherence and statistical leverage , 2011, ICML.

[18]  Timo Eirola,et al.  On Smooth Decompositions of Matrices , 1999, SIAM J. Matrix Anal. Appl..

[19]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[20]  Damien Stehlé,et al.  Rigorous Perturbation Bounds of Some Matrix Factorizations , 2010, SIAM J. Matrix Anal. Appl..

[21]  Gary L. Miller,et al.  Iterative Row Sampling , 2012, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[22]  Ji-guang Sun.  Componentwise perturbation bounds for some matrix decompositions , 1992 .

[23]  P. Wedin.  Perturbation bounds in connection with singular value decomposition , 1972 .

[24]  G. Stewart.  On the perturbation of LU, Cholesky, and QR factorizations , 1993 .

[25]  H. Zha.  A componentwise perturbation analysis of the QR decomposition , 1993 .

[26]  Ilse C. F. Ipsen,et al.  The Effect of Coherence on Sampling from Matrices with Orthonormal Columns, and Preconditioned Least Squares Problems , 2014, SIAM J. Matrix Anal. Appl..

[27]  R. Bhatia,et al.  Variation of the Unitary Part of a Matrix , 1994 .

[28]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[29]  Xiao-Wen Chang,et al.  Componentwise perturbation analyses for the QR factorization , 2001, Numerische Mathematik.

[30]  Ji-guang Sun.  On perturbation bounds for the QR factorization , 1995 .

[31]  N. Higham.  Computing the polar decomposition with applications , 1986 .

[32]  G. Stewart,et al.  Perturbation Analyses for the QR Factorization , 1997, SIAM J. Matrix Anal. Appl..

[33]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[34]  S. Muthukrishnan,et al.  Subspace Sampling and Relative-Error Matrix Approximation: Column-Based Methods , 2006, APPROX-RANDOM.