Constructing Fast Approximate Eigenspaces With Application to the Fast Graph Fourier Transforms

We investigate numerically efficient approximations of eigenspaces associated with symmetric and general matrices. The eigenspaces are factored into a fixed number of fundamental components that can be efficiently manipulated which we consider to be extended orthogonal Givens or scaling and shear transformations. The number of these components controls the trade-off between approximation accuracy and the computational complexity of projecting on the eigenspaces. We write minimization problems for the single fundamental components and provide closed-form solutions. Then we propose algorithms that iteratively update all these components until convergence. We show results on random matrices and an application on the approximation of graph Fourier transforms for directed and undirected graphs.

[1]  Ann B. Lee,et al.  Treelets--An adaptive multi-scale basis for sparse unordered data , 2007, 0707.0481.

[2]  Nicolas Tremblay,et al.  Approximate Fast Graph Fourier Transforms via Multilayer Sparse Approximations , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[3]  G. Golub,et al.  Eigenvalue computation in the 20th century , 2000 .

[4]  A Díaz-Guilera,et al.  Self-similar community structure in a network of human interactions. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[5]  W. Givens.  Computation of Plain Unitary Rotations Transforming a General Matrix to Triangular Form , 1958 .

[6]  Joan Bruna,et al.  Approximating Orthogonal Matrices with Effective Givens Factorization , 2019, ICML.

[7]  Vikas K. Garg,et al.  Multiresolution Matrix Factorization , 2014, ICML.

[8]  G. W. Stewart,et al.  The decompositional approach to matrix computation , 2000, Comput. Sci. Eng..

[9]  Cristian Rusu,et al.  Learning Multiplication-free Linear Transformations , 2018, Digit. Signal Process..

[10]  S. L. Wong,et al.  Towards a proteome-scale map of the human protein–protein interaction network , 2005, Nature.

[11]  C. Jacobi.  Über ein leichtes Verfahren die in der Theorie der Säcularstörungen vorkommenden Gleichungen numerisch aufzulösen*). , 2022 .

[12]  J. Tukey,et al.  An algorithm for the machine calculation of complex Fourier series , 1965 .

[13]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[14]  P. Schönemann,et al.  On two-sided orthogonal procrustes problems , 1968, Psychometrika.

[15]  P. Henrici.  On the Speed of Convergence of Cyclic and Quasicyclic Jacobi Methods for Computing Eigenvalues of Hermitian Matrices , 1958 .

[16]  Lorenzo Rosasco,et al.  Fast approximation of orthogonal matrices and application to PCA , 2019, Signal Process..

[17]  W. Gander,et al.  A Constrained Eigenvalue Problem , 1989 .

[18]  Gal Chechik,et al.  Coordinate-descent for learning orthogonal matrices through Givens rotations , 2014, ICML.

[19]  Risi Kondor,et al.  Asymmetric Multiresolution Matrix Factorization , 2019, ArXiv.

[20]  Cristian Rusu,et al.  Learning Fast Sparsifying Transforms , 2016, IEEE Transactions on Signal Processing.

[21]  Amir Averbuch,et al.  Randomized LU Decomposition , 2013, ArXiv.

[22]  Sushant Sachdeva,et al.  Approximate Gaussian Elimination for Laplacians - Fast, Sparse, and Simple , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[23]  I. Daubechies,et al.  Factoring wavelet transforms into lifting steps , 1998 .

[24]  Jure Leskovec,et al.  Learning to Discover Social Circles in Ego Networks , 2012, NIPS.

[25]  J. Meijerink,et al.  An iterative solution method for linear systems of which the coefficient matrix is a symmetric -matrix , 1977 .