Multi-scale Wasserstein Shortest-path Graph Kernels for Graph Classification

Graph kernels are conventional methods for computing graph similarities. However, most of the R-convolution graph kernels face two challenges: 1) They cannot compare graphs at multiple different scales, and 2) they do not consider the distributions of substructures when computing the kernel matrix. These two challenges limit their performances. To mitigate the two challenges, we propose a novel graph kernel called the Multi-scale Path-pattern Graph kernel (MPG), at the heart of which is the multi-scale path-pattern node feature map. Each element of the path-pattern node feature map is the number of occurrences of a path-pattern around a node. A path-pattern is constructed by the concatenation of all the node labels in a path of a truncated BFS tree rooted at each node. Since the path-pattern node feature map can only compare graphs at local scales, we incorporate into it the multiple different scales of the graph structure, which are captured by the truncated BFS trees of different depth. We use the Wasserstein distance to compute the similarity between the multi-scale path-pattern node feature maps of two graphs, considering the distributions of path-patterns. We empirically validate MPG on various benchmark graph datasets and demonstrate that it achieves state-of-the-art performance.

[1]  Till Hendrik Schulz,et al.  Graph Filtration Kernels , 2021, AAAI.

[2]  Ambuj K. Singh,et al.  A Broader Picture of Random-walk Based Graph Embedding , 2021, KDD.

[3]  K. Borgwardt,et al.  Filtration Curves for Graph Representation , 2021, KDD.

[4]  Sheng Li,et al.  Co-Embedding of Nodes and Edges With Graph Neural Networks , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Claudia Plant,et al.  Data Compression as a Comprehensive Framework for Graph Drawing and Representation Learning , 2020, KDD.

[6]  Ambuj K. Singh,et al.  Tree++: Truncated Tree Based Graph Kernels , 2020, IEEE Transactions on Knowledge and Data Engineering.

[7]  Lu Bai,et al.  A Hierarchical Transitive-Aligned Graph Kernel for Un-attributed Graphs , 2020, ICML.

[8]  Jieping Ye,et al.  PINE: Universal Deep Embedding for Graph Nodes via Partial Permutation Invariant Set Functions , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Karsten M. Borgwardt,et al.  Wasserstein Weisfeiler-Lehman Graph Kernels , 2019, NeurIPS.

[10]  Ruosong Wang,et al.  Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernels , 2019, NeurIPS.

[11]  Ruosong Wang,et al.  On Exact Computation with an Infinitely Wide Neural Net , 2019, NeurIPS.

[12]  Yijian Xiang,et al.  RetGK: Graph Kernels based on Return Probabilities of Random Walks , 2018, NeurIPS.

[13]  Charu C. Aggarwal,et al.  Learning Deep Network Representations with Adversarially Regularized Autoencoders , 2018, KDD.

[14]  Philip S. Yu,et al.  Deep Recursive Network Embedding with Regular Equivalence , 2018, KDD.

[15]  Wenwu Zhu,et al.  Deep Variational Network Embedding in Wasserstein Space , 2018, KDD.

[16]  Arthur Jacot,et al.  Neural Tangent Kernel: Convergence and Generalization in Neural Networks , 2018, NeurIPS.

[17]  Michalis Vazirgiannis,et al.  GraKeL: A Graph Kernel Library in Python , 2018, J. Mach. Learn. Res..

[18]  Nils M. Kriege,et al.  Recognizing Cuneiform Signs Using Graph Based Methods , 2018, COST@SDM.

[19]  Jian Pei,et al.  A Survey on Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.

[20]  Jian Li,et al.  Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec , 2017, WSDM.

[21]  Kevin Chen-Chuan Chang,et al.  A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[22]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..

[23]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[24]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[25]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[26]  Chengqi Zhang,et al.  Task Sensitive Feature Exploration and Learning for Multitask Graph Classification , 2017, IEEE Transactions on Cybernetics.

[27]  Michalis Vazirgiannis,et al.  Matching Node Embeddings for Graph Similarity , 2017, AAAI.

[28]  Jian Pei,et al.  Community Preserving Network Embedding , 2017, AAAI.

[29]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[30]  Jian Pei,et al.  Asymmetric Transitivity Preserving Graph Embedding , 2016, KDD.

[31]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[32]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[33]  Nils M. Kriege,et al.  On Valid Optimal Assignment Kernels and Applications to Graph Classification , 2016, NIPS.

[34]  Cheng Soon Ong,et al.  Learning SVM in Kreĭn Spaces , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Risi Kondor,et al.  The Multiscale Laplacian Graph Kernel , 2016, NIPS.

[36]  Wei Lu,et al.  Deep Neural Networks for Learning Graph Representations , 2016, AAAI.

[37]  R. Garnett,et al.  Propagation kernels: efficient graph kernels from propagated information , 2016, Machine Learning.

[38]  Qiongkai Xu,et al.  GraRep: Learning Graph Representations with Global Structural Information , 2015, CIKM.

[39]  Pinar Yanardag,et al.  Deep Graph Kernels , 2015, KDD.

[40]  Deli Zhao,et al.  Network Representation Learning with Rich Text Information , 2015, IJCAI.

[41]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[42]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[43]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[44]  Roman Garnett,et al.  Graph Kernels for Object Category Prediction in Task-Dependent Robot Grasping , 2013, MLG 2013.

[45]  Jean-Charles Delvenne,et al.  The stability of a graph partition: A dynamics-based framework for community detection , 2013, ArXiv.

[46]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[47]  Nils M. Kriege,et al.  Subgraph Matching Kernels for Attributed Graphs , 2012, ICML.

[48]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[49]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[50]  Kurt Mehlhorn,et al.  Weisfeiler-Lehman Graph Kernels , 2011, J. Mach. Learn. Res..

[51]  Fabrizio Costa,et al.  Fast Neighborhood Subgraph Pairwise Distance Kernel , 2010, ICML.

[52]  Karsten M. Borgwardt,et al.  Fast subtree kernels on graphs , 2009, NIPS.

[53]  Kurt Mehlhorn,et al.  Efficient graphlet kernels for large graph comparison , 2009, AISTATS.

[54]  Jean-Charles Delvenne,et al.  Stability of graph communities across time scales , 2008, Proceedings of the National Academy of Sciences.

[55]  C. Villani Optimal Transport: Old and New , 2008 .

[56]  Karsten M. Borgwardt,et al.  Graph Kernels , 2008, J. Mach. Learn. Res..

[57]  Fan Chung,et al.  The heat kernel as the pagerank of a graph , 2007, Proceedings of the National Academy of Sciences.

[58]  Trevor Darrell,et al.  The Pyramid Match Kernel: Efficient Learning with Sets of Features , 2007, J. Mach. Learn. Res..

[59]  George Karypis,et al.  Comparison of descriptor spaces for chemical compound retrieval and classification , 2006, Sixth International Conference on Data Mining (ICDM'06).

[60]  Kevin J. Lang,et al.  Local Graph Partitioning using PageRank Vectors , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[61]  Jean-Philippe Vert,et al.  Graph kernels based on tree patterns for molecules , 2006, Machine Learning.

[62]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[63]  Hans-Peter Kriegel,et al.  Shortest-path kernels on graphs , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[64]  Claus Bahlmann,et al.  Learning with Distance Substitution Kernels , 2004, DAGM-Symposium.

[65]  Thomas Gärtner,et al.  Cyclic pattern kernels for predictive graph mining , 2004, KDD.

[66]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[67]  Jeffrey J. Sutherland,et al.  Spline-Fitting with a Genetic Algorithm: A Method for Developing Classification Structure-Activity Relationships , 2003, J. Chem. Inf. Comput. Sci..

[68]  Tony Jebara,et al.  A Kernel Between Sets of Vectors , 2003, ICML.

[69]  P. Dobson,et al.  Distinguishing enzyme structures from non-enzymes without alignments. , 2003, Journal of molecular biology.

[70]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[71]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[72]  Herbert Edelsbrunner,et al.  Topological Persistence and Simplification , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[73]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[74]  A. Debnath,et al.  Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. Correlation with molecular orbital energies and hydrophobicity. , 1991, Journal of medicinal chemistry.

[75]  Michael C. Hout,et al.  Multidimensional Scaling , 2003, Encyclopedic Dictionary of Archaeology.

[76]  Alessandro Sperduti,et al.  A Tree-Based Kernel for Graphs , 2012, SDM.

[77]  Cédric Villani,et al.  Optimal Transport and Curvature , 2011 .

[78]  Hans-Peter Kriegel,et al.  Protein function prediction via graph kernels , 2005, ISMB.

[79]  Thomas Gärtner,et al.  On Graph Kernels: Hardness Results and Efficient Alternatives , 2003, COLT.

[80]  Jan Ramon,et al.  Expressivity versus efficiency of graph kernels , 2003 .

[81]  David Haussler,et al.  Convolution kernels on discrete structures , 1999 .