Learning Laplacian Matrix in Smooth Graph Signal Representations

The construction of a meaningful graph plays a crucial role in the success of many graph-based representations and algorithms for handling structured data, especially in the emerging field of graph signal processing. However, a meaningful graph is not always readily available from the data, nor easy to define depending on the application domain. In particular, it is often desirable in graph signal processing applications that a graph is chosen such that the data admit certain regularity or smoothness on the graph. In this paper, we address the problem of learning graph Laplacians, which is equivalent to learning graph topologies, such that the input data form graph signals with smooth variations on the resulting topology. To this end, we adopt a factor analysis model for the graph signals and impose a Gaussian probabilistic prior on the latent variables that control these signals. We show that the Gaussian prior leads to an efficient representation that favors the smoothness property of the graph signals. We then propose an algorithm for learning graphs that enforces such property and is based on minimizing the variations of the signals on the learned graph. Experiments on both synthetic and real world data demonstrate that the proposed graph learning framework can efficiently infer meaningful graph topologies from signal observations under the smoothness prior.

[1]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[2]  Dimitri Van De Ville,et al.  Wavelet frames on graphs defined by fMRI functional connectivity , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[3]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs , 2012, IEEE Transactions on Signal Processing.

[4]  David J. Bartholomew,et al.  Latent Variable Models and Factor Analysis: A Unified Approach , 2011 .

[5]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[6]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[7]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[8]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[9]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[10]  Alexander Basilevsky,et al.  Statistical Factor Analysis and Related Methods , 1994 .

[11]  Pierre Vandergheynst,et al.  A windowed graph Fourier transform , 2012, 2012 IEEE Statistical Signal Processing Workshop (SSP).

[12]  Daniel A. Spielman,et al.  Fitting a graph to vector data , 2009, ICML '09.

[13]  Cha Zhang,et al.  Analyzing the Optimality of Predictive Transform Coding Using Graph-Based Models , 2013, IEEE Signal Processing Letters.

[14]  Adam J. Rothman,et al.  Sparse permutation invariant covariance estimation , 2008, 0801.4837.

[15]  Michael G. Rabbat,et al.  Approximating signals supported on graphs , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[17]  B. Schölkopf,et al.  A Regularization Framework for Learning from Graph Data , 2004, ICML 2004.

[18]  Dit-Yan Yeung,et al.  Kernel selection forl semi-supervised kernel machines , 2007, ICML '07.

[19]  Yue M. Lu,et al.  A Spectral Graph Uncertainty Principle , 2012, IEEE Transactions on Information Theory.

[20]  A. Singer From graph to manifold Laplacian: The convergence rate , 2006 .

[21]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[22]  Tony Jebara,et al.  Laplacian Spectrum Learning , 2010, ECML/PKDD.

[23]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[24]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Ronald R. Coifman,et al.  Multiscale Wavelets on Trees, Graphs and High Dimensional Data: Theory and Applications to Semi Supervised Learning , 2010, ICML.

[26]  Pascal Frossard,et al.  Learning Parametric Dictionaries for Signals on Graphs , 2014, IEEE Transactions on Signal Processing.

[27]  Quanzheng Li,et al.  A graph theoretical regression model for brain connectivity learning of Alzheimer'S disease , 2013, 2013 IEEE 10th International Symposium on Biomedical Imaging.

[28]  Bin Yu,et al.  Model Selection in Gaussian Graphical Models: High-Dimensional Consistency of boldmathell_1-regularized MLE , 2008, NIPS 2008.

[29]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[30]  Pierre Vandergheynst,et al.  Spectrum-Adapted Tight Graph Wavelet and Vertex-Frequency Frames , 2013, IEEE Transactions on Signal Processing.

[31]  Joshua B. Tenenbaum,et al.  Discovering Structure by Learning Sparse Graphs , 2010 .

[32]  Quanzheng Li,et al.  Matched Signal Detection on Graphs: Theory and Application to Brain Network Classification , 2013, IPMI.

[33]  Mark Herbster,et al.  Combining Graph Laplacians for Semi-Supervised Learning , 2005, NIPS.

[34]  Jaejoon Lee,et al.  Edge-adaptive transforms for efficient depth map coding , 2010, 28th Picture Coding Symposium.

[35]  Zoubin Ghahramani,et al.  Semi-supervised learning : from Gaussian fields to Gaussian processes , 2003 .

[36]  Pascal Frossard,et al.  Inference of mobility patterns via Spectral Graph Wavelets , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[37]  Sam T. Roweis,et al.  EM Algorithms for PCA and Sensible PCA , 1997, NIPS 1997.

[38]  Po-Ling Loh,et al.  Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses , 2012, NIPS.

[39]  B. Bollobás The evolution of random graphs , 1984 .

[40]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[41]  P. Erdos,et al.  On the evolution of random graphs , 1984 .

[42]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[43]  Pradeep Ravikumar,et al.  Sparse inverse covariance matrix estimation using quadratic approximation , 2011, MLSLP.

[44]  Martin J. Wainwright,et al.  Model Selection in Gaussian Graphical Models: High-Dimensional Consistency of l1-regularized MLE , 2008, NIPS.

[45]  Alexander J. Smola,et al.  Kernels and Regularization on Graphs , 2003, COLT.

[46]  Inderjit S. Dhillon,et al.  Metric and Kernel Learning Using a Linear Transformation , 2009, J. Mach. Learn. Res..

[47]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[48]  Ulrike von Luxburg,et al.  From Graphs to Manifolds - Weak and Strong Pointwise Consistency of Graph Laplacians , 2005, COLT.

[49]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[50]  Sunil K. Narang,et al.  Perfect Reconstruction Two-Channel Wavelet Filter Banks for Graph Structured Data , 2011, IEEE Transactions on Signal Processing.

[51]  Pascal Frossard,et al.  Laplacian matrix learning for smooth graph signal representation , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[52]  Mikhail Belkin,et al.  Towards a theoretical foundation for Laplacian-based manifold methods , 2005, J. Comput. Syst. Sci..

[53]  Pascal Frossard,et al.  Learning of structured graph dictionaries , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[54]  Pierre Vandergheynst,et al.  Vertex-Frequency Analysis on Graphs , 2013, ArXiv.

[55]  Dimitri Van De Ville,et al.  Principal components of functional connectivity: A new approach to study dynamic brain connectivity during rest , 2013, NeuroImage.

[56]  Pierre Borgnat,et al.  Graph Wavelets for Multiscale Community Mining , 2014, IEEE Transactions on Signal Processing.

[57]  Pascal Frossard,et al.  Clustering With Multi-Layer Graphs: A Spectral Perspective , 2011, IEEE Transactions on Signal Processing.

[58]  Rémi Gribonval,et al.  Should Penalized Least Squares Regression be Interpreted as Maximum A Posteriori Estimation? , 2011, IEEE Transactions on Signal Processing.

[59]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[60]  Michael Elad,et al.  Generalized Tree-Based Wavelet Transform , 2010, IEEE Transactions on Signal Processing.

[61]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[62]  Leonhard Held,et al.  Gaussian Markov Random Fields: Theory and Applications , 2005 .

[63]  Mikhail Belkin,et al.  Towards a Theoretical Foundation for Laplacian-Based Manifold Methods , 2005, COLT.

[64]  R. Coifman,et al.  Diffusion Wavelets , 2004 .