A survey on graph kernels

Graph kernels have become an established and widely-used technique for solving classification tasks on graphs. This survey gives a comprehensive overview of techniques for kernel-based graph classification developed in the past 15 years. We describe and categorize graph kernels based on properties inherent to their design, such as the nature of their extracted graph features, their method of computation and their applicability to problems in practice. In an extensive experimental evaluation, we study the classification accuracy of a large suite of graph kernels on established benchmarks as well as new datasets. We compare the performance of popular kernels with several baseline methods and study the effect of applying a Gaussian RBF kernel to the metric induced by a graph kernel. In doing so, we find that simple baselines become competitive after this transformation on some datasets. Moreover, we study the extent to which existing graph kernels agree in their predictions (and prediction errors) and obtain a data-driven categorization of kernels as result. Finally, based on our experimental results, we derive a practitioner’s guide to kernel-based graph classification.

[1]  Bernhard Schölkopf,et al.  Kernel Principal Component Analysis , 1997, ICANN.

[2]  Tetsuji Kuboyama,et al.  A generalization of Haussler's convolution kernel: mapping kernel , 2008, ICML.

[3]  Cheng Soon Ong,et al.  Learning SVM in Kreĭn Spaces , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Ryan A. Rossi,et al.  Estimation of local subgraph counts , 2016, 2016 IEEE International Conference on Big Data (Big Data).

[5]  Yi Zhang,et al.  A Comprehensive Evaluation of Graph Kernels for Unattributed Graphs , 2018, Entropy.

[6]  Melanie Hilario,et al.  Adaptive Matching Based Kernels for Labelled Graphs , 2010, PAKDD.

[7]  László Lovász,et al.  On the Shannon capacity of a graph , 1979, IEEE Trans. Inf. Theory.

[8]  Daisuke Kihara,et al.  Protein Function Prediction , 2017, Methods in Molecular Biology.

[9]  Hans-Peter Kriegel,et al.  Graph Kernels For Disease Outcome Prediction From Protein-Protein Interaction Networks , 2006, Pacific Symposium on Biocomputing.

[10]  Xifeng Yan,et al.  A Fast Kernel for Attributed Graphs , 2016, SDM.

[11]  S. V. N. Vishwanathan,et al.  A Structural Smoothing Framework For Robust Graph Comparison , 2015, NIPS.

[12]  Trevor Darrell,et al.  The Pyramid Match Kernel: Efficient Learning with Sets of Features , 2007, J. Mach. Learn. Res..

[13]  Ameet Talwalkar,et al.  Foundations of Machine Learning , 2012, Adaptive computation and machine learning.

[14]  Zaïd Harchaoui,et al.  Image Classification with Segmentation Graph Kernels , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[16]  Chengqi Zhang,et al.  Nested Subtree Hash Kernels for Large-Scale Graph Classification over Streams , 2012, 2012 IEEE 12th International Conference on Data Mining.

[17]  Jean-Philippe Vert,et al.  The Pharmacophore Kernel for Virtual Screening with Support Vector Machines , 2006, J. Chem. Inf. Model..

[18]  Gerben de Vries.  A Fast Approximation of the Weisfeiler-Lehman Graph Kernel for RDF Data , 2013, ECML/PKDD.

[19]  Antje Chang,et al.  New Developments , 2003 .

[20]  Heinrich Müller,et al.  SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Pinar Yanardag,et al.  Deep Graph Kernels , 2015, KDD.

[22]  Michalis Vazirgiannis,et al.  Enhancing Graph Kernels via Successive Embeddings , 2018, CIKM.

[23]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[24]  Karsten M. Borgwardt,et al.  The graphlet spectrum , 2009, ICML '09.

[25]  Kaspar Riesen,et al.  IAM Graph Database Repository for Graph Based Pattern Recognition and Machine Learning , 2008, SSPR/SPR.

[26]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..

[27]  Kristian Kersting,et al.  Glocalized Weisfeiler-Lehman Graph Kernels: Global-Local Feature Maps of Graphs , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[28]  S. V. N. Vishwanathan,et al.  Graph kernels , 2007 .

[29]  Tatsuya Akutsu,et al.  Graph Kernels for Molecular Structure-Activity Relationship Analysis with Support Vector Machines , 2005, J. Chem. Inf. Model..

[30]  Edwin R. Hancock,et al.  fMRI Activation Network Analysis Using Bose-Einstein Entropy , 2016, S+SSPR.

[31]  Alessio Ceroni,et al.  Classification of small molecules by two- and three-dimensional decomposition kernels , 2007, Bioinform..

[32]  Mahantapas Kundu,et al.  The journey of graph kernels through two decades , 2018, Comput. Sci. Rev..

[33]  Devdatt P. Dubhashi,et al.  Entity disambiguation in anonymized graphs using graph kernels , 2013, CIKM.

[34]  Martin Schäf,et al.  Detecting Similar Programs via The Weisfeiler-Leman Graph Kernel , 2016, ICSR.

[35]  Roman Garnett,et al.  Propagation kernels: efficient graph kernels from propagated information , 2015, Machine Learning.

[36]  Kristian Kersting,et al.  Explicit Versus Implicit Graph Feature Maps: A Computational Phase Transition for Walk Kernels , 2014, 2014 IEEE International Conference on Data Mining.

[37]  Devdatt P. Dubhashi,et al.  Learning with Similarity Functions on Graphs using Matchings of Geometric Embeddings , 2015, KDD.

[38]  Chunfeng Yuan,et al.  Human Action Recognition Based on Context-Dependent Graph Kernels , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  George W. Adamson,et al.  A method for the automatic classification of chemical structures , 1973, Inf. Storage Retr..

[40]  Hans-Peter Kriegel,et al.  Protein function prediction via graph kernels , 2005, ISMB.

[41]  Brendan D. McKay,et al.  Practical graph isomorphism, II , 2013, J. Symb. Comput..

[42]  Vikas Singh,et al.  Solving the multi-way matching problem by permutation synchronization , 2013, NIPS.

[43]  Jan Ramon,et al.  Frequent subgraph mining in outerplanar graphs , 2006, KDD '06.

[44]  Sandro Vega-Pons,et al.  Brain Decoding via Graph Kernels , 2013, 2013 International Workshop on Pattern Recognition in Neuroimaging.

[45]  Nils M. Kriege.  Deep Weisfeiler-Lehman assignment kernels via multiple kernel learning , 2019, ESANN.

[46]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[47]  Ashwin Srinivasan,et al.  The Predictive Toxicology Challenge 2000-2001 , 2001, Bioinform..

[48]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[49]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[50]  Michalis Vazirgiannis,et al.  Matching Node Embeddings for Graph Similarity , 2017, AAAI.

[51]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[52]  J. Kazius,et al.  Derivation and validation of toxicophores for mutagenicity prediction. , 2005, Journal of medicinal chemistry.

[53]  J. Fleiss.  Measuring nominal scale agreement among many raters. , 1971 .

[54]  Andreas Zell,et al.  Optimal assignment kernels for attributed molecular graphs , 2005, ICML.

[55]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[56]  Aaron Roth,et al.  The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..

[57]  Maurice Bruynooghe,et al.  A polynomial time computable metric between point sets , 2001, Acta Informatica.

[58]  Kristian Kersting,et al.  A unifying view of explicit and implicit feature maps of graph kernels , 2017, Data Mining and Knowledge Discovery.

[59]  George Karypis,et al.  Comparison of descriptor spaces for chemical compound retrieval and classification , 2006, Sixth International Conference on Data Mining (ICDM'06).

[60]  Andrea Torsello,et al.  Transitive Assignment Kernels for Structural Classification , 2015, SIMBAD.

[61]  Pierre Baldi,et al.  Kernels for small molecules and the prediction of mutagenicity, toxicity and anti-cancer activity , 2005, ISMB.

[62]  Alessandro Sperduti,et al.  A memory efficient graph kernel , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[63]  David Rogers,et al.  Extended-Connectivity Fingerprints , 2010, J. Chem. Inf. Model..

[64]  Felix Naumann,et al.  Data fusion , 2009, CSUR.

[65]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[66]  Sylvain Bernard,et al.  The 2.1 Ga Old Francevillian Biota: Biogenicity, Taphonomy and Biodiversity , 2014, PloS one.

[67]  Hans-Peter Kriegel,et al.  Shortest-path kernels on graphs , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[68]  Christian Sohler,et al.  A Property Testing Framework for the Theoretical Expressivity of Graph Kernels , 2018, IJCAI.

[69]  Karsten M. Borgwardt,et al.  Halting in Random Walk Kernels , 2015, NIPS.

[70]  Bertrand Thirion,et al.  Graph-Based Inter-Subject Pattern Analysis of fMRI Data , 2014, PloS one.

[71]  Roman Garnett,et al.  Graph Kernels for Object Category Prediction in Task-Dependent Robot Grasping , 2013, MLG 2013.

[72]  Jeffrey J. Sutherland,et al.  Spline-Fitting with a Genetic Algorithm: A Method for Developing Classification Structure-Activity Relationships , 2003, J. Chem. Inf. Comput. Sci..

[73]  David Haussler,et al.  Convolution kernels on discrete structures , 1999 .

[74]  Alessandro Sperduti,et al.  Measuring the expressivity of graph kernels through Statistical Learning Theory , 2017, Neurocomputing.

[75]  Tatsuya Akutsu,et al.  Extensions of marginalized graph kernels , 2004, ICML.

[76]  Kurt Mehlhorn,et al.  Weisfeiler-Lehman Graph Kernels , 2011, J. Mach. Learn. Res..

[77]  Yannis Stavrakas,et al.  Shortest-Path Graph Kernels for Document Similarity , 2017, EMNLP.

[78]  Hisashi Kashima,et al.  A Linear-Time Graph Kernel , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[79]  Jean-Philippe Vert,et al.  Graph kernels based on tree patterns for molecules , 2006, Machine Learning.

[80]  Osamu Watanabe,et al.  Generalized Shortest Path Kernel on Graphs , 2015, Discovery Science.

[81]  Devdatt P. Dubhashi,et al.  Classifying Large Graphs with Differential Privacy , 2015, MDAI.

[82]  Trevor Darrell,et al.  Approximate Correspondences in High Dimensions , 2006, NIPS.

[83]  Yijian Xiang,et al.  RetGK: Graph Kernels based on Return Probabilities of Random Walks , 2018, NeurIPS.

[84]  Alessandro Sperduti,et al.  Hyper-Parameter Tuning for Graph Kernels via Multiple Kernel Learning , 2016, ICONIP.

[85]  Nils M. Kriege,et al.  Subgraph Matching Kernels for Attributed Graphs , 2012, ICML.

[86]  Jimeng Sun,et al.  Fast Random Walk Graph Kernel , 2012, SDM.

[87]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[88]  A. Debnath,et al.  Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. Correlation with molecular orbital energies and hydrophobicity. , 1991, Journal of medicinal chemistry.

[89]  Pierre Baldi,et al.  Graph kernels for chemical informatics , 2005, Neural Networks.

[90]  Ravi Kumar,et al.  Counting Graphlets: Space vs Time , 2017, WSDM.

[91]  Risi Kondor,et al.  The Multiscale Laplacian Graph Kernel , 2016, NIPS.

[92]  Nils M. Kriege,et al.  On Valid Optimal Assignment Kernels and Applications to Graph Classification , 2016, NIPS.

[93]  László Babai,et al.  Canonical labelling of graphs in linear average time , 1979, 20th Annual Symposium on Foundations of Computer Science (sfcs 1979).

[94]  Edwin R. Hancock,et al.  A Graph Kernel from the Depth-Based Representation , 2014, S+SSPR.

[95]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[96]  Atsuko Yamaguchi,et al.  Graph Complexity of Chemical Compounds in Biological Pathways , 2003 .

[97]  Hisashi Kashima,et al.  Marginalized Kernels Between Labeled Graphs , 2003, ICML.

[98]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[99]  P. Dobson,et al.  Distinguishing enzyme structures from non-enzymes without alignments. , 2003, Journal of molecular biology.

[100]  David S. Johnson,et al.  The NP-completeness column , 2005, TALG.

[101]  Martin Grohe,et al.  Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks , 2018, AAAI.

[102]  Thomas Lengauer,et al.  Automatic Generation of Complementary Descriptors with Molecular Graph Networks , 2005, J. Chem. Inf. Model..

[103]  Thomas Gärtner,et al.  On Graph Kernels: Hardness Results and Efficient Alternatives , 2003, COLT.

[104]  Marion Neumann.  Learning with Graphs using Kernels from Propagated Information , 2015 .

[105]  Kristian Kersting,et al.  Faster Kernels for Graphs with Continuous Attributes via Hashing , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[106]  Michalis Vazirgiannis,et al.  A Degeneracy Framework for Graph Similarity , 2018, IJCAI.

[107]  Kurt Mehlhorn,et al.  Efficient graphlet kernels for large graph comparison , 2009, AISTATS.

[108]  Edwin R. Hancock,et al.  A transitive aligned Weisfeiler-Lehman subtree kernel , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[109]  Jure Leskovec,et al.  Hierarchical Graph Representation Learning with Differentiable Pooling , 2018, NeurIPS.

[110]  P. Willett,et al.  A Comparison of Some Measures for the Determination of Inter‐Molecular Structural Similarity Measures of Inter‐Molecular Structural Similarity , 1986 .

[111]  Jan Ramon,et al.  Expressivity versus efficiency of graph kernels , 2003 .

[112]  Marleen de Bruijne,et al.  Scalable kernels for graphs with continuous attributes , 2013, NIPS.

[113]  Maria-Florina Balcan,et al.  On a theory of learning with similarity functions , 2006, ICML.

[114]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[115]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[116]  N. Shervashidze.  Scalable graph kernels , 2012 .

[117]  Hanghang Tong,et al.  Cheetah: Fast Graph Kernel Tracking on Dynamic Graphs , 2015, SDM.

[118]  John C. S. Lui,et al.  Mining Graphlet Counts in Online Social Networks , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[119]  Alessandro Sperduti,et al.  Multiple Graph-Kernel Learning , 2015, 2015 IEEE Symposium Series on Computational Intelligence.

[120]  Alessandro Sperduti,et al.  A Tree-Based Kernel for Graphs , 2012, SDM.

[121]  James G. Nourse,et al.  Reoptimization of MDL Keys for Use in Drug Discovery , 2002, J. Chem. Inf. Comput. Sci..

[122]  Fabrizio Costa,et al.  Fast Neighborhood Subgraph Pairwise Distance Kernel , 2010, ICML.

[123]  Thomas Gärtner,et al.  Cyclic pattern kernels for predictive graph mining , 2004, KDD.

[124]  Jean-Philippe Vert,et al.  The optimal assignment kernel is not positive definite , 2008, ArXiv.

[125]  Edwin R. Hancock,et al.  An Aligned Subtree Kernel for Weighted Graphs , 2015, ICML.

[126]  BaldiPierre,et al.  2005 Speical Issue , 2005 .

[127]  Nathan Brown,et al.  Chemoinformatics—an introduction for computer scientists , 2009, CSUR.

[128]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[129]  Luc De Raedt,et al.  Graph Invariant Kernels , 2015, IJCAI.

[130]  Yixin Chen,et al.  An End-to-End Deep Learning Architecture for Graph Classification , 2018, AAAI.

[131]  Daoqiang Zhang,et al.  Sub-network Based Kernels for Brain Network Classification , 2016, BCB.

[132]  Devdatt P. Dubhashi,et al.  Global graph kernels using geometric embeddings , 2014, ICML.

[133]  Sandro Vega-Pons,et al.  Classification of inter-subject fMRI data based on graph kernels , 2014, 2014 International Workshop on Pattern Recognition in Neuroimaging.