Approximate Fast Graph Fourier Transforms via Multilayer Sparse Approximations

The fast Fourier transform is an algorithm of paramount importance in signal processing as it allows to apply the Fourier transform in <inline-formula><tex-math notation="LaTeX">$\mathcal {O}(n \log n)$</tex-math></inline-formula> instead of <inline-formula><tex-math notation="LaTeX">$\mathcal {O}(n^2)$</tex-math></inline-formula> arithmetic operations. Graph signal processing is a recent research domain that generalizes classical signal processing tools, such as the Fourier transform, to situations where the signal domain is given by any arbitrary graph instead of a regular grid. Today, there is no method to rapidly apply graph Fourier transforms. In this paper, we propose a method to obtain approximate graph Fourier transforms that can be applied rapidly and stored efficiently. It is based on a greedy approximate diagonalization of the graph Laplacian matrix, carried out using a modified version of the famous Jacobi eigenvalues algorithm. The method is described and analyzed in detail, and then applied to both synthetic and real graphs, showing its potential.

[1]  Xin Wang,et al.  On Algorithms for Sparse Multi-factor NMF , 2013, NIPS.

[2]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[3]  Michael Elad,et al.  Trainlets: Dictionary Learning in High Dimensions , 2016, IEEE Transactions on Signal Processing.

[4]  G. Golub,et al.  Eigenvalue computation in the 20th century , 2000 .

[5]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs: Frequency Analysis , 2013, IEEE Transactions on Signal Processing.

[6]  Alston S. Householder,et al.  Unitary Triangularization of a Nonsymmetric Matrix , 1958, JACM.

[7]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs , 2012, IEEE Transactions on Signal Processing.

[8]  Laurent Demanet,et al.  Fast Computation of Fourier Integral Operators , 2006, SIAM J. Sci. Comput..

[9]  Jean-Yves Tourneret,et al.  Optimization of a Fast Transform Structured as a Convolutional Tree , 2016 .

[10]  Cristopher Moore,et al.  Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[11]  Don Coppersmith,et al.  Matrix multiplication via arithmetic progressions , 1987, STOC.

[12]  Bing Zeng,et al.  Design of non-separable transforms for directional 2-D sources , 2011, 2011 18th IEEE International Conference on Image Processing.

[13]  R. Brent,et al.  The Solution of Singular-Value and Symmetric Eigenvalue Problems on Multiprocessor Arrays , 1985 .

[14]  Rémi Gribonval,et al.  Flexible Multilayer Sparse Approximations of Matrices and Applications , 2015, IEEE Journal of Selected Topics in Signal Processing.

[15]  Jacques Morgenstern,et al.  The Linear Complexity of Computation , 1975, JACM.

[16]  José M. F. Moura,et al.  Big Data Analysis with Signal Processing on Graphs: Representation and processing of massive data sets with irregular structure , 2014, IEEE Signal Processing Magazine.

[17]  Pierre Vandergheynst,et al.  Compressive Spectral Clustering , 2016, ICML.

[18]  R. Coifman,et al.  Fast wavelet transforms and numerical algorithms I , 1991 .

[19]  Robert W. Heath,et al.  Fast Orthonormal Sparsifying Transforms Based on Householder Reflectors , 2016, IEEE Transactions on Signal Processing.

[20]  V. Strassen Gaussian elimination is not optimal , 1969 .

[21]  Rémi Gribonval,et al.  Are there approximate fast fourier transforms on graphs? , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[22]  Wolfgang Hackbusch,et al.  A Sparse Matrix Arithmetic Based on H-Matrices. Part I: Introduction to H-Matrices , 1999, Computing.

[23]  J. Tukey,et al.  An algorithm for the machine calculation of complex Fourier series , 1965 .

[24]  C. Jacobi Über ein leichtes Verfahren die in der Theorie der Säcularstörungen vorkommenden Gleichungen numerisch aufzulösen*). , 2022 .

[25]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[26]  Vikas K. Garg,et al.  Multiresolution Matrix Factorization , 2014, ICML.

[27]  Risi Kondor,et al.  Multiresolution Matrix Compression , 2016, AISTATS.

[28]  Michael Elad,et al.  Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation , 2010, IEEE Transactions on Signal Processing.

[29]  Wen-Hsiung Chen,et al.  A Fast Computational Algorithm for the Discrete Cosine Transform , 1977, IEEE Trans. Commun..

[30]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[31]  Jean-Yves Tourneret,et al.  Toward Fast Transform Learning , 2014, International Journal of Computer Vision.

[32]  Charles A. Bouman,et al.  The Sparse Matrix Transform for Covariance Estimation and Analysis of High Dimensional Signals , 2011, IEEE Transactions on Image Processing.

[33]  Bing Zeng,et al.  Design of low-complexity, non-separable 2-D transforms based on butterfly structures , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[34]  T. Dupont,et al.  Polynomial approximation of functions in Sobolev spaces , 1980 .

[35]  Ann B. Lee,et al.  Treelets--An adaptive multi-scale basis for sparse unordered data , 2007, 0707.0481.

[36]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[37]  W. Givens Computation of Plain Unitary Rotations Transforming a General Matrix to Triangular Form , 1958 .

[38]  Pierre Vandergheynst,et al.  GSPBOX: A toolbox for signal processing on graphs , 2014, ArXiv.

[39]  Elwood S. Buffa,et al.  Graph Theory with Applications , 1977 .

[40]  V. Rokhlin Rapid solution of integral equations of classical potential theory , 1985 .

[41]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[42]  C. Jacobi,et al.  C. G. J. Jacobi's Gesammelte Werke: Über ein leichtes Verfahren, die in der Theorie der Sacularstorungen vorkommenden Gleichungen numerisch aufzulosen , 1846 .

[43]  François Le Gall,et al.  Powers of tensors and fast matrix multiplication , 2014, ISSAC.