Applications of Trace Estimation Techniques

We discuss various applications of trace estimation techniques for evaluating functions of the form \(\mathtt {tr}(f(A))\) where f is certain function. The first problem we consider that can be cast in this form is that of approximating the Spectral density or Density of States (DOS) of a matrix. The DOS is a probability density distribution that measures the likelihood of finding eigenvalues of the matrix at a given point on the real line, and it is an important function in solid state physics. We also present a few non-standard applications of spectral densities. Other trace estimation problems we discuss include estimating the trace of a matrix inverse \(\mathtt {tr}(A^{-1})\), the problem of counting eigenvalues and estimating the rank, and approximating the log-determinant (trace of log function). We also discuss a few similar computations that arise in machine learning applications. We review two computationally inexpensive methods to compute traces of matrix functions, namely, the Chebyshev expansion and the Lanczos Quadrature methods. A few numerical examples are presented to illustrate the performances of these methods in different applications.

[1]  David P. Woodruff,et al.  On Sketching Matrix Norms and the Top Singular Vector , 2014, SODA.

[2]  Leonhard Held,et al.  Gaussian Markov Random Fields: Theory and Applications , 2005 .

[3]  I. Turek.  A maximum-entropy approach to the density of states within the recursion method , 1988 .

[4]  G. Golub,et al.  Some large-scale matrix computation problems , 1996 .

[5]  M. Hutchinson.  A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines , 1989 .

[6]  Ernesto Estrada.  Characterization of 3D molecular structure , 2000 .

[7]  Y. Zhang,et al.  Approximate implementation of the logarithm of the matrix determinant in Gaussian process regression , 2007 .

[8]  T. J. Rivlin.  The Chebyshev polynomials , 1974 .

[9]  Mihai Anitescu,et al.  Computing f(A)b via Least Squares Polynomial Approximations , 2011, SIAM J. Sci. Comput..

[10]  R. Silver,et al.  DENSITIES OF STATES OF MEGA-DIMENSIONAL HAMILTONIAN MATRICES , 1994 .

[11]  M. Newman,et al.  Finding community structure in networks using the eigenvectors of matrices. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[12]  David P. Woodruff,et al.  Spectrum Approximation Beyond Fast Matrix Multiplication: Algorithms and Hardness , 2017, ITCS.

[13]  M. ScholarWorks.  Estimating the trace of the matrix inverse by interpolating from the diagonal of an approximate inverse , 2019 .

[14]  Timothy A. Davis,et al.  The university of Florida sparse matrix collection , 2011, TOMS.

[15]  Wang,et al.  Calculating the density of states and optical-absorption spectra of large quantum systems by the plane-wave moments method. , 1994, Physical review. B, Condensed matter.

[16]  Nicholas J. Higham,et al.  Functions of matrices - theory and computation , 2008 .

[17]  Christos Boutsidis,et al.  A Randomized Algorithm for Approximating the Log Determinant of a Symmetric Positive Definite Matrix , 2015, ArXiv.

[18]  D. Botstein,et al.  Singular value decomposition for genome-wide expression data processing and modeling. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[19]  R. Carbó-Dorca.  Smooth function topological structure descriptors based on graph-spectra , 2008 .

[20]  Y. Saad,et al.  An estimator for the diagonal of a matrix , 2007 .

[21]  Uri M. Ascher,et al.  Improved Bounds on Sample Size for Implicit Matrix Trace Estimators , 2013, Found. Comput. Math..

[22]  Gene H. Golub,et al.  Estimates in quadratic formulas , 1994, Numerical Algorithms.

[23]  Gene H. Golub,et al.  Calculation of Gauss quadrature rules , 1967, Milestones in Matrix Computation.

[24]  Ali S. Hadi,et al.  Finding Groups in Data: An Introduction to Chster Analysis , 1991 .

[25]  G. Wellein,et al.  The kernel polynomial method , 2005, cond-mat/0504627.

[26]  Jinwoo Shin,et al.  Approximating Spectral Sums of Large-Scale Matrices using Stochastic Chebyshev Approximations , 2017, SIAM J. Sci. Comput..

[27]  Edoardo Di Napoli,et al.  Efficient estimation of eigenvalue counts in an interval , 2013, Numer. Linear Algebra Appl..

[28]  J. Mason,et al.  Integration Using Chebyshev Polynomials , 2003 .

[29]  Jinwoo Shin,et al.  Large-scale log-determinant computation through stochastic Chebyshev expansions , 2015, ICML.

[30]  Yousef Saad,et al.  Fast Estimation of Approximate Matrix Ranks Using Spectral Densities , 2016, Neural Computation.

[31]  Sivan Toledo,et al.  Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix , 2011, JACM.

[32]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[33]  Yousef Saad,et al.  Fast Estimation of tr(f(A)) via Stochastic Lanczos Quadrature , 2017, SIAM J. Matrix Anal. Appl..

[34]  Jo Eidsvik,et al.  Parameter estimation in high dimensional Gaussian distributions , 2011, Stat. Comput..

[35]  Martin J. Wainwright,et al.  Distributed Estimation of Generalized Matrix Rank: Efficient Algorithms and Lower Bounds , 2015, ICML.

[36]  Yousef Saad,et al.  Approximating Spectral Densities of Large Matrices , 2013, SIAM Rev..

[37]  Yousef Saad,et al.  Fast Computation of Spectral Densities for Generalized Eigenvalue Problems , 2017, SIAM J. Sci. Comput..

[38]  Alexandr Andoni,et al.  SKETCHING AND EMBEDDING ARE EQUIVALENT FOR , 2018 .

[39]  Yousef Saad,et al.  Fast methods for estimating the Numerical rank of large matrices , 2016, ICML.

[40]  G. Golub,et al.  Matrices, Moments and Quadrature with Applications , 2009 .

[41]  Constantine Bekas,et al.  Accelerating data uncertainty quantification by solving linear systems with multiple right-hand sides , 2013, Numerical Algorithms.