Gaussian Processes on Graphs Via Spectral Kernel Learning

We propose a graph spectrum-based Gaussian process for prediction of signals defined on nodes of the graph. The model is designed to capture various graph signal structures through a highly adaptive kernel that incorporates a flexible polynomial function in the graph spectral domain. Unlike most existing approaches, we propose to learn such a spectral kernel, where the polynomial setup enables learning without the need for eigen-decomposition of the graph Laplacian. In addition, this kernel has the interpretability of graph filtering achieved by a bespoke maximum likelihood learning algorithm that enforces the positivity of the spectrum. We demonstrate the interpretability of the model in synthetic experiments from which we show the various ground truth spectral filters can be accurately recovered, and the adaptability translates to superior performances in the prediction of real-world graph data of various characteristics.

[1]  Alexander J. Smola,et al.  Kernels and Regularization on Graphs , 2003, COLT.

[2]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[3]  Bernhard Schölkopf,et al.  From Regularization Operators to Support Vector Kernels , 1997, NIPS.

[4]  Dimitri Van De Ville,et al.  Signal-Adapted Tight Frames on Graphs , 2016, IEEE Transactions on Signal Processing.

[5]  Pierre Vandergheynst,et al.  Graph Signal Processing: Overview, Challenges, and Applications , 2017, Proceedings of the IEEE.

[6]  Andrew Gordon Wilson,et al.  Gaussian Process Kernels for Pattern Discovery and Extrapolation , 2013, ICML.

[7]  Pascal Frossard,et al.  Learning Parametric Dictionaries for Signals on Graphs , 2014, IEEE Transactions on Signal Processing.

[8]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[9]  Yin Cheng Ng,et al.  Bayesian Semi-supervised Learning with Graph Gaussian Processes , 2018, NeurIPS.

[10]  Xiaowen Dong,et al.  Kernel-Based Graph Learning From Smooth Signals: A Functional Viewpoint , 2020, IEEE Transactions on Signal and Information Processing over Networks.

[11]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[12]  L. H. Loomis.  An Introduction to Abstract Harmonic Analysis , 1953 .

[13]  Yong Liu,et al.  Automated Spectral Kernel Learning , 2020, AAAI.

[14]  Ian Walker,et al.  Graph Convolutional Gaussian Processes , 2019, ICML.

[15]  Carl E. Rasmussen,et al.  Convolutional Gaussian Processes , 2017, NIPS.

[16]  Alexander Skabardonis,et al.  Freeway Performance Measurement System: Operational Analysis Tool , 2002 .

[17]  Pascal Frossard,et al.  Parametric dictionary learning for graph signals , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[18]  Peter Händel,et al.  Gaussian Processes Over Graphs , 2018, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[19]  Rafail N. Gasimov,et al.  Augmented Lagrangian Duality and Nondifferentiable Optimization Methods in Nonconvex Programming , 2002, J. Glob. Optim..

[20]  Yao Hu,et al.  Uncertainty Aware Graph Gaussian Process for Semi-Supervised Learning , 2020, AAAI.

[21]  Alexander J. Smola,et al.  Regularization with Dot-Product Kernels , 2000, NIPS.

[22]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[23]  Pietro Lio,et al.  Graph Convolutional Gaussian Processes For Link Prediction , 2020, ArXiv.

[24]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[25]  Yves-Laurent Kom Samo,et al.  Generalized Spectral Kernels , 2015, 1506.02236.

[26]  Peter Händel,et al.  Kernel Regression for Signals over Graphs , 2017, ArXiv.

[27]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[28]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[29]  José M. F. Moura,et al.  Signal denoising on graphs via graph filtering , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[30]  M. Bazaraa,et al.  A survey of various tactics for generating Lagrangian multipliers in the context of Lagrangian duality , 1979 .

[31]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[32]  Neil D. Lawrence,et al.  Kernels for Vector-Valued Functions: a Review , 2011, Found. Trends Mach. Learn..

[33]  E. Bozzo.  The Moore–Penrose inverse of the normalized graph Laplacian , 2013 .