Graph Learning From Data Under Laplacian and Structural Constraints

Graphs are fundamental mathematical structures used in various fields to represent data, signals, and processes. In this paper, we propose a novel framework for learning/estimating graphs from data. The proposed framework includes (i) formulation of various graph learning problems, (ii) their probabilistic interpretations, and (iii) associated algorithms. Specifically, graph learning problems are posed as the estimation of graph Laplacian matrices from some observed data under given structural constraints (e.g., graph connectivity and sparsity level). From a probabilistic perspective, the problems of interest correspond to maximum a posteriori parameter estimation of Gaussian–Markov random field models, whose precision (inverse covariance) is a graph Laplacian matrix. For the proposed graph learning problems, specialized algorithms are developed by incorporating the graph Laplacian and structural constraints. The experimental results demonstrate that the proposed algorithms outperform the current state-of-the-art methods in terms of accuracy and computational efficiency.

[1]  Daniel A. Spielman Algorithms, Graph Theory, and the Solution of Laplacian Linear Equations , 2012, ICALP.

[2]  L. N. Vicente,et al.  A comparison of block pivoting and interior-point algorithms for linear least squares problems with nonnegative variables , 1994 .

[3]  Antonio Ortega,et al.  Generalized Laplacian precision matrix estimation for graph signal processing , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Shang-Hua Teng,et al.  Nearly-Linear Time Algorithms for Preconditioning and Solving Symmetric, Diagonally Dominant Linear Systems , 2006, SIAM J. Matrix Anal. Appl..

[5]  Sergio Barbarossa,et al.  Graph topology inference based on transform learning , 2016, 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[6]  Trevor J. Hastie,et al.  Exact Covariance Thresholding into Connected Components for Large-Scale Graphical Lasso , 2011, J. Mach. Learn. Res..

[7]  Vassilis Kalofolias,et al.  How to Learn a Graph from Smooth Signals , 2016, AISTATS.

[8]  G. Poole,et al.  A Survey on M-Matrices , 1974 .

[9]  Florian Dörfler,et al.  Kron Reduction of Graphs With Applications to Electrical Networks , 2011, IEEE Transactions on Circuits and Systems I: Regular Papers.

[10]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[11]  Antonio Ortega,et al.  Graph Learning from Data under Structural and Laplacian Constraints , 2016, ArXiv.

[12]  Stephen J. Wright Coordinate descent algorithms , 2015, Mathematical Programming.

[13]  秀俊 松井,et al.  Statistics for High-Dimensional Data: Methods, Theory and Applications , 2014 .

[14]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[15]  Philip A. Chou,et al.  Graph Signal Processing – A Probabilistic Framework , 2016 .

[16]  Antonio Ortega,et al.  Graph learning with Laplacian constraints: Modeling attractive Gaussian Markov random fields , 2016, 2016 50th Asilomar Conference on Signals, Systems and Computers.

[17]  Antonio Ortega,et al.  GTT: Graph template transforms with applications to image coding , 2015, 2015 Picture Coding Symposium (PCS).

[18]  Antonio Ortega,et al.  GBST: Separable transforms based on line graphs for predictive video coding , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[19]  Charles Kemp,et al.  The discovery of structural form , 2008, Proceedings of the National Academy of Sciences.

[20]  Inderjit S. Dhillon,et al.  Matrix Nearness Problems with Bregman Divergences , 2007, SIAM J. Matrix Anal. Appl..

[21]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[22]  Trevor J. Hastie,et al.  The Graphical Lasso: New Insights and Alternatives , 2011, Electronic journal of statistics.

[23]  A. Murat Tekalp,et al.  Digital Video Processing , 1995 .

[24]  Ulrike von Luxburg,et al.  The f-Adjusted Graph Laplacian: a Diagonal Modification with a Geometric Interpretation , 2014, ICML.

[25]  Santiago Segarra,et al.  Network Topology Inference from Spectral Templates , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[26]  Sunil K. Narang,et al.  Perfect Reconstruction Two-Channel Wavelet Filter Banks for Graph Structured Data , 2011, IEEE Transactions on Signal Processing.

[27]  J. Sherman,et al.  Adjustment of an Inverse Matrix Corresponding to a Change in One Element of a Given Matrix , 1950 .

[28]  Shang-Hua Teng,et al.  Spectral sparsification of graphs: theory and algorithms , 2013, CACM.

[29]  Peyman Milanfar,et al.  A Tour of Modern Image Filtering: New Insights and Methods, Both Practical and Theoretical , 2013, IEEE Signal Processing Magazine.

[30]  Peter F. Stadler,et al.  Laplacian Eigenvectors of Graphs , 2007 .

[31]  Shang-Hua Teng,et al.  A Local Clustering Algorithm for Massive Graphs and Its Application to Nearly Linear Time Graph Partitioning , 2008, SIAM J. Comput..

[32]  Joshua B. Tenenbaum,et al.  Discovering Structure by Learning Sparse Graphs , 2010 .

[33]  Leonhard Held,et al.  Gaussian Markov Random Fields: Theory and Applications , 2005 .

[34]  Michael G. Rabbat,et al.  Characterization and Inference of Graph Diffusion Processes From Observations of Stationary Signals , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[35]  Su-In Lee,et al.  Pathway Graphical Lasso , 2015, AAAI.

[36]  Nir Friedman,et al.  Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning , 2009 .

[37]  J. S. Rao,et al.  Spike and slab variable selection: Frequentist and Bayesian strategies , 2005, math/0505633.

[38]  M. Wainwright,et al.  HIGH-DIMENSIONAL COVARIANCE ESTIMATION BY MINIMIZING l1-PENALIZED LOG-DETERMINANT DIVERGENCE BY PRADEEP RAVIKUMAR , 2009 .

[39]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[40]  Cha Zhang,et al.  Analyzing the Optimality of Predictive Transform Coding Using Graph-Based Models , 2013, IEEE Signal Processing Letters.

[41]  Mihailo R. Jovanovic,et al.  Topology identification of undirected consensus networks via sparse inverse covariance estimation , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[42]  Matthias Hein,et al.  Estimation of positive definite M-matrices and structure learning for attractive Gaussian Markov Random fields , 2014, 1404.6640.

[43]  Min Xu,et al.  High-dimensional Covariance Estimation Based On Gaussian Graphical Models , 2010, J. Mach. Learn. Res..

[44]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[45]  Anil K. Jain Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[46]  Pascal Frossard,et al.  Learning Laplacian Matrix in Smooth Graph Signal Representations , 2014, IEEE Transactions on Signal Processing.

[47]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[48]  Alexander J. Smola,et al.  Kernels and Regularization on Graphs , 2003, COLT.

[49]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[50]  Antonio Ortega,et al.  Spectral anomaly detection using graph-based filtering for wireless sensor networks , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[51]  Donghui Chen,et al.  Nonnegativity constraints in numerical analysis , 2009, The Birth of Numerical Analysis.

[52]  T. Cover,et al.  Determinant inequalities via information theory , 1988 .

[53]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[54]  Po-Ling Loh,et al.  Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses , 2012, NIPS.

[55]  Antonio Ortega,et al.  Submitted to Ieee Transactions on Signal Processing 1 Efficient Sampling Set Selection for Bandlimited Graph Signals Using Graph Spectral Proxies , 2022 .

[56]  Pradeep Ravikumar,et al.  BIG & QUIC: Sparse Inverse Covariance Estimation for a Million Variables , 2013, NIPS.

[57]  Amir Beck,et al.  On the Convergence of Block Coordinate Descent Type Methods , 2013, SIAM J. Optim..