Distribution of Node Embeddings as Multiresolution Features for Graphs

Graph classification is an important problem in many fields, from bioinformatics and neuroscience to computer vision and social network analysis. That said, the task of comparing graphs for the purpose of graph classification faces several major challenges. In particular, an effective graph comparison method must (1) expressively and inductively compare graphs; (2) efficiently compare large graphs; and (3) enable the use of fast machine learning models for graph classification. To address such challenges, we propose Randomized Grid Mapping (RGM), a fast-to-compute feature map that represents a graph via the distribution of its node embeddings in feature space. We justify RGM with close connections to kernel methods: RGM provably approximates the Laplacian kernel mean map and has the multiresolution properties of the pyramid match kernel. We also show that RGM can be extended to incorporate node labels using the Weisfeiler-Lehman framework. Extensive experiments show that graph classification accuracy with RGM feature maps is better than or competitive with many powerful graph kernels, unsupervised graph feature mappings, and deep neural networks. Moreover, comparing graphs based on their node embeddings with RGM is up to an order of magnitude faster than competitive baselines, while maintaining high classification accuracy.

[1]  Danai Koutra,et al.  Network similarity via multiple social theories , 2013, 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2013).

[2]  Kurt Mehlhorn,et al.  Efficient graphlet kernels for large graph comparison , 2009, AISTATS.

[3]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[4]  Jure Leskovec,et al.  Hierarchical Graph Representation Learning with Differentiable Pooling , 2018, NeurIPS.

[5]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[6]  Emmanuel Müller,et al.  NetLSD: Hearing the Shape of a Graph , 2018, KDD.

[7]  Tina Eliassi-Rad,et al.  A Guide to Selecting a Network Similarity Method , 2014, SDM.

[8]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[9]  Danai Koutra,et al.  Exploratory Analysis of Graph Data by Leveraging Domain Knowledge , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[10]  Kristian Kersting,et al.  Glocalized Weisfeiler-Lehman Graph Kernels: Global-Local Feature Maps of Graphs , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[11]  Kristian Kersting,et al.  Faster Kernels for Graphs with Continuous Attributes via Hashing , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[12]  Danai Koutra,et al.  Individual and Collective Graph Mining: Principles, Algorithms, and Applications , 2017, Individual and Collective Graph Mining.

[13]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[14]  Roman Garnett,et al.  Propagation kernels: efficient graph kernels from propagated information , 2015, Machine Learning.

[15]  Donald F. Towsley,et al.  Diffusion-Convolutional Neural Networks , 2015, NIPS.

[16]  Michalis Vazirgiannis,et al.  Matching Node Embeddings for Graph Similarity , 2017, AAAI.

[17]  Petros Drineas,et al.  On the Nyström Method for Approximating a Gram Matrix for Improved Kernel-Based Learning , 2005, J. Mach. Learn. Res..

[18]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[19]  Sergey Ivanov,et al.  Anonymous Walk Embeddings , 2018, ICML.

[20]  Jean-Philippe Vert,et al.  Graph kernels based on tree patterns for molecules , 2006, Machine Learning.

[21]  Pinar Yanardag,et al.  Deep Graph Kernels , 2015, KDD.

[22]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[23]  Nils M. Kriege,et al.  On Valid Optimal Assignment Kernels and Applications to Graph Classification , 2016, NIPS.

[24]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[25]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[26]  Yao Zhang,et al.  Sub2Vec: Feature Learning for Subgraphs , 2018, PAKDD.

[27]  Benjamin Recht,et al.  Random Features for Large-Scale Kernel Machines , 2007, NIPS.

[28]  David B. Blumenthal,et al.  On the exact computation of the graph edit distance , 2020, Pattern Recognit. Lett..

[29]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[30]  Danai Koutra,et al.  From Community to Role-based Graph Embeddings , 2019, ArXiv.

[31]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[32]  Kurt Mehlhorn,et al.  Weisfeiler-Lehman Graph Kernels , 2011, J. Mach. Learn. Res..

[33]  Yijian Xiang,et al.  RetGK: Graph Kernels based on Return Probabilities of Random Walks , 2018, NeurIPS.

[34]  Mark Heimann.  On Generalizing Neural Node Embedding Methods to Multi-Network Problems , 2017 .

[35]  Trevor Darrell,et al.  The Pyramid Match Kernel: Efficient Learning with Sets of Features , 2007, J. Mach. Learn. Res..

[36]  Danai Koutra,et al.  GroupINN: Grouping-based Interpretable Neural Network for Classification of Limited, Noisy Brain Data , 2019, KDD.

[37]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[38]  Thorsten Joachims,et al.  Training linear SVMs in linear time , 2006, KDD '06.

[39]  Danai Koutra,et al.  Latent Network Summarization: Bridging Network Embedding and Summarization , 2018, KDD.

[40]  Charu C. Aggarwal,et al.  Scalable Global Alignment Graph Kernel Using Random Features: From Node Embedding to Graph Embedding , 2019, KDD.

[41]  Zhi-Li Zhang,et al.  Hunt For The Unique, Stable, Sparse And Fast Feature Learning On Graphs , 2017, NIPS.

[42]  Devdatt P. Dubhashi,et al.  Learning with Similarity Functions on Graphs using Matchings of Geometric Embeddings , 2015, KDD.

[43]  Hans-Peter Kriegel,et al.  Shortest-path kernels on graphs , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[44]  Le Song,et al.  A Hilbert Space Embedding for Distributions , 2007, Discovery Science.

[45]  Yixin Chen,et al.  An End-to-End Deep Learning Architecture for Graph Classification , 2018, AAAI.

[46]  Mark Heimann,et al.  REGAL: Representation Learning-based Graph Alignment , 2018, CIKM.

[47]  Mark Heimann,et al.  HashAlign: Hash-Based Alignment of Multiple Graphs , 2018, PAKDD.

[48]  Danai Koutra,et al.  DELTACON: A Principled Massive-Graph Similarity Function , 2013, SDM.

[49]  Kristian Kersting,et al.  Explicit Versus Implicit Graph Feature Maps: A Computational Phase Transition for Walk Kernels , 2014, 2014 IEEE International Conference on Data Mining.

[50]  Risi Kondor,et al.  The Multiscale Laplacian Graph Kernel , 2016, NIPS.