Hunt For The Unique, Stable, Sparse And Fast Feature Learning On Graphs

For the purpose of learning on graphs, we hunt for a graph feature representation that exhibit certain uniqueness, stability and sparsity properties while also being amenable to fast computation. This leads to the discovery of family of graph spectral distances (denoted as FGSD) and their based graph feature representations, which we prove to possess most of these desired properties. To both evaluate the quality of graph features produced by FGSD and demonstrate their utility, we apply them to the graph classification problem. Through extensive experiments, we show that a simple SVM based classification algorithm, driven with our powerful FGSD based graph features, significantly outperforms all the more sophisticated state-of-art algorithms on the unlabeled node datasets in terms of both accuracy and speed; it also yields very competitive results on the labeled datasets - despite the fact it does not utilize any node label information.

[1]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[2]  Hans-Peter Kriegel,et al.  Shortest-path kernels on graphs , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[3]  Ronald R. Coifman,et al.  Diffusion Maps, Spectral Clustering and Eigenfunctions of Fokker-Planck Operators , 2005, NIPS.

[4]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[5]  László Babai,et al.  Graph isomorphism in quasipolynomial time [extended abstract] , 2015, STOC.

[6]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[7]  J. Bunch,et al.  Rank-one modification of the symmetric eigenproblem , 1978 .

[8]  Risi Kondor,et al.  The Multiscale Laplacian Graph Kernel , 2016, NIPS.

[9]  Karsten M. Borgwardt,et al.  The graphlet spectrum , 2009, ICML '09.

[10]  L. Shapiro An Electrical Lemma , 1987 .

[11]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[12]  André Elisseeff,et al.  Stability and Generalization , 2002, J. Mach. Learn. Res..

[13]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[14]  T. D. Morley,et al.  Eigenvalues of the Laplacian of a graph , 1985 .

[15]  P. Seymour,et al.  The Structure of Homometric Sets , 1982 .

[16]  Kurt Mehlhorn,et al.  Efficient graphlet kernels for large graph comparison , 2009, AISTATS.

[17]  Luc De Raedt,et al.  Graph Invariant Kernels , 2015, IJCAI.

[18]  F. Spieksma,et al.  Effective graph resistance , 2011 .

[19]  Kurt Mehlhorn,et al.  Weisfeiler-Lehman Graph Kernels , 2011, J. Mach. Learn. Res..

[20]  Colin McDiarmid,et al.  Surveys in Combinatorics, 1989: On the method of bounded differences , 1989 .

[21]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[22]  Thomas Gärtner,et al.  On Graph Kernels: Hardness Results and Efficient Alternatives , 2003, COLT.

[23]  Donald F. Towsley,et al.  Diffusion-Convolutional Neural Networks , 2015, NIPS.

[24]  Pinar Yanardag,et al.  Deep Graph Kernels , 2015, KDD.

[25]  Karsten M. Borgwardt,et al.  The skew spectrum of graphs , 2008, ICML '08.

[26]  Thomas A. Funkhouser,et al.  Biharmonic distance , 2010, TOGS.

[27]  Joan Bruna,et al.  Deep Convolutional Networks on Graph-Structured Data , 2015, ArXiv.

[28]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[29]  David Bommes,et al.  Efficient Linear System Solvers for Mesh Processing , 2005, IMA Conference on the Mathematics of Surfaces.

[30]  Edmund Taylor Whittaker,et al.  A Course of Modern Analysis , 2021 .

[31]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[32]  Risi Kondor A complete set of rotationally and translationally invariant features for images , 2007, ArXiv.

[33]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[34]  Vasilios N. Katsikis,et al.  An improved method for the computation of the Moore-Penrose inverse matrix , 2011, Appl. Math. Comput..

[35]  Le Song,et al.  Discriminative Embeddings of Latent Variable Models for Structured Data , 2016, ICML.

[36]  Fabrizio Costa,et al.  Fast Neighborhood Subgraph Pairwise Distance Kernel , 2010, ICML.

[37]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[38]  Andreas Ziehe,et al.  Learning Invariant Representations of Molecules for Atomization Energy Prediction , 2012, NIPS.

[39]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.