Learning Sparse Graph Laplacian with K Eigenvector Prior via Iterative GLASSO and Projection

Learning a suitable graph is an important precursor to many graph signal processing (GSP) pipelines, such as graph spectral signal compression and denoising. Previous graph learning algorithms either i) make some assumptions on connectivity (e.g., graph sparsity), or ii) make simple graph edge assumptions such as positive edges only. In this paper, given an empirical covariance matrix $\bar{C}$ computed from data as input, we consider a structural assumption on the graph Laplacian matrix $L$: the first $K$ eigenvectors of $L$ are pre-selected, e.g., based on domain-specific criteria, such as computation requirement, and the remaining eigenvectors are then learned from data. One example use case is image coding, where the first eigenvector is pre-chosen to be constant, regardless of available observed data. We first prove that the subspace of symmetric positive semi-definite (PSD) matrices $H_{u}^+$ with the first $K$ eigenvectors being $\{u_k\}$ in a defined Hilbert space is a convex cone. We then construct an operator to project a given positive definite (PD) matrix $L$ to $H_{u}^+$, inspired by the Gram-Schmidt procedure. Finally, we design an efficient hybrid graphical lasso/projection algorithm to compute the most suitable graph Laplacian matrix $L^* \in H_{u}^+$ given $\bar{C}$. Experimental results show that given the first $K$ eigenvectors as a prior, our algorithm outperforms competing graph learning schemes using a variety of graph comparison metrics.

[1]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[2]  Andrew V. Knyazev,et al.  Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method , 2001, SIAM J. Sci. Comput..

[3]  Trevor J. Hastie,et al.  The Graphical Lasso: New Insights and Alternatives , 2011, Electronic journal of statistics.

[4]  Stephen J. Wright Coordinate descent algorithms , 2015, Mathematical Programming.

[5]  Ping Zhu,et al.  A study of graph spectra for comparing graphs and trees , 2008, Pattern Recognit..

[6]  Walter Hoffmann,et al.  Iterative algorithms for Gram-Schmidt orthogonalization , 1989, Computing.

[7]  Wen Gao,et al.  Graph-Based Blind Image Deblurring From a Single Photograph , 2018, IEEE Transactions on Image Processing.

[8]  Rémi Gribonval,et al.  Analyzing the approximation error of the fast graph Fourier transform , 2017, 2017 51st Asilomar Conference on Signals, Systems, and Computers.

[9]  Pierre Vandergheynst,et al.  Graph Signal Processing: Overview, Challenges, and Applications , 2017, Proceedings of the IEEE.

[10]  Alan M. Frieze,et al.  Random graphs , 2006, SODA '06.

[11]  Vivek K Goyal,et al.  Foundations of Signal Processing , 2014 .

[12]  Oscar C. Au,et al.  Optimal graph laplacian regularization for natural image denoising , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[14]  Gene Cheung,et al.  Graph Sampling for Matrix Completion Using Recurrent Gershgorin Disc Shift , 2019, IEEE Transactions on Signal Processing.

[15]  Antonio Ortega,et al.  Graph Learning From Data Under Laplacian and Structural Constraints , 2016, IEEE Journal of Selected Topics in Signal Processing.

[16]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Maximum Likelihood Estimation , 2007, ArXiv.

[17]  T. Cai,et al.  A Constrained ℓ1 Minimization Approach to Sparse Precision Matrix Estimation , 2011, 1102.2233.

[18]  Gene Cheung,et al.  Point Cloud Denoising via Feature Graph Laplacian Regularization , 2020, IEEE Transactions on Image Processing.

[19]  W. Wallis,et al.  A Graph-Theoretic Approach to Enterprise Network Dynamics , 2006 .

[20]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[21]  Pascal Frossard,et al.  Learning Laplacian Matrix in Smooth Graph Signal Representations , 2014, IEEE Transactions on Signal Processing.

[22]  Enrico Magli,et al.  Graph Spectral Image Processing , 2018, Proceedings of the IEEE.

[23]  Oscar C. Au,et al.  Multiresolution Graph Fourier Transform for Compression of Piecewise Smooth Images , 2015, IEEE Transactions on Image Processing.

[24]  Francesca Ieva,et al.  Comparing methods for comparing networks , 2019, Scientific Reports.

[25]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[26]  Danai Koutra,et al.  DELTACON: A Principled Massive-Graph Similarity Function , 2013, SDM.

[27]  Gene Cheung,et al.  Robust Semisupervised Graph Classifier Learning With Negative Edge Weights , 2016, IEEE Transactions on Signal and Information Processing over Networks.