Approximating Orthogonal Matrices with Effective Givens Factorization

We analyze effective approximation of unitary matrices. In our formulation, a unitary matrix is represented as a product of rotations in two-dimensional subspaces, so-called Givens rotations. Instead of the quadratic dimension dependence when applying a dense matrix, applying such an approximation scales with the number factors, each of which can be implemented efficiently. Consequently, in settings where an approximation is once computed and then applied many times, such a representation becomes advantageous. Although effective Givens factorization is not possible for generic unitary operators, we show that minimizing a sparsity-inducing objective with a coordinate descent algorithm on the unitary group yields good factorizations for structured matrices. Canonical applications of such a setup are orthogonal basis transforms. We demonstrate numerical results of approximating the graph Fourier transform, which is the matrix obtained when diagonalizing a graph Laplacian.

[1]  Nicolas Tremblay,et al.  Approximate Fast Graph Fourier Transforms via Multilayer Sparse Approximations , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[2]  G. Golub,et al.  Eigenvalue computation in the 20th century , 2000 .

[3]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[4]  J. Tukey,et al.  An algorithm for the machine calculation of complex Fourier series , 1965 .

[5]  M. V. Wilkes,et al.  The Art of Computer Programming, Volume 3, Sorting and Searching , 1974 .

[6]  Sushant Sachdeva,et al.  Approximate Gaussian Elimination for Laplacians - Fast, Sparse, and Simple , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[7]  Gal Chechik,et al.  Coordinate-descent for learning orthogonal matrices through Givens rotations , 2014, ICML.

[8]  S. L. Wong,et al.  Towards a proteome-scale map of the human protein–protein interaction network , 2005, Nature.

[9]  Arnold Schönhage,et al.  Zur quadratischen Konvergenz des Jacobi-Verfahrens , 1964 .

[10]  Jack J. Dongarra,et al.  Guest Editors Introduction to the top 10 algorithms , 2000, Comput. Sci. Eng..

[11]  W. Givens Computation of Plain Unitary Rotations Transforming a General Matrix to Triangular Form , 1958 .

[12]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[13]  B. Hall Lie Groups, Lie Algebras, and Representations , 2003 .

[14]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[15]  Stanford,et al.  Learning to Discover Social Circles in Ego Networks , 2012 .

[16]  A Díaz-Guilera,et al.  Self-similar community structure in a network of human interactions. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[17]  C. Jacobi Über ein leichtes Verfahren die in der Theorie der Säcularstörungen vorkommenden Gleichungen numerisch aufzulösen*). , 2022 .

[18]  Vikas K. Garg,et al.  Multiresolution Matrix Factorization , 2014, ICML.