Graph Kernels: A Survey

Graph kernels have attracted a lot of attention during the last decade, and have evolved into a rapidly developing branch of learning on structured data. During the past 20 years, the considerable research activity that occurred in the field resulted in the development of dozens of graph kernels, each focusing on specific structural properties of graphs. Graph kernels have proven successful in a wide range of domains, ranging from social networks to bioinformatics. The goal of this survey is to provide a unifying view of the literature on graph kernels. In particular, we present a comprehensive overview of a wide range of graph kernels. Furthermore, we perform an experimental evaluation of several of those kernels on publicly available datasets, and provide a comparative study. Finally, we discuss key applications of graph kernels, and outline some challenges that remain to be addressed.

[1]  Karsten Borgwardt,et al.  Graph Kernels: State-of-the-Art and Future Challenges , 2020, Found. Trends Mach. Learn..

[2]  Julien Mairal,et al.  Convolutional Kernel Networks for Graph-Structured Data , 2020, ICML.

[3]  A. Micheli,et al.  A Fair Comparison of Graph Neural Networks for Graph Classification , 2019, ICLR.

[4]  Petra Mutzel,et al.  Weisfeiler and Leman go sparse: Towards scalable higher-order graph embeddings , 2019, NeurIPS.

[5]  Nils M. Kriege,et al.  A survey on graph kernels , 2019, Applied Network Science.

[6]  Michalis Vazirgiannis,et al.  GraKeL: A Graph Kernel Library in Python , 2018, J. Mach. Learn. Res..

[7]  Michalis Vazirgiannis,et al.  Random Walk Graph Neural Networks , 2020, NeurIPS.

[8]  Karsten M. Borgwardt,et al.  Wasserstein Weisfeiler-Lehman Graph Kernels , 2019, NeurIPS.

[9]  Ruosong Wang,et al.  Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernels , 2019, NeurIPS.

[10]  Karsten M. Borgwardt,et al.  A Persistent Weisfeiler-Lehman Procedure for Graph Classification , 2019, ICML.

[11]  Bryan Perozzi,et al.  DDGK: Learning Graph Representations for Deep Divergence Graph Kernels , 2019, WWW.

[12]  Vinayak A. Rao,et al.  Relational Pooling for Graph Representations , 2019, ICML.

[13]  Martin Grohe,et al.  Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks , 2018, AAAI.

[14]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[15]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Alessandro Sperduti,et al.  Pre-training Graph Neural Networks with Kernels , 2018, ArXiv.

[17]  Michalis Vazirgiannis,et al.  Enhancing Graph Kernels via Successive Embeddings , 2018, CIKM.

[18]  Yijian Xiang,et al.  RetGK: Graph Kernels based on Return Probabilities of Random Walks , 2018, NeurIPS.

[19]  Christian Sohler,et al.  A Property Testing Framework for the Theoretical Expressivity of Graph Kernels , 2018, IJCAI.

[20]  Michalis Vazirgiannis,et al.  A Degeneracy Framework for Graph Similarity , 2018, IJCAI.

[21]  Jure Leskovec,et al.  Hierarchical Graph Representation Learning with Differentiable Pooling , 2018, NeurIPS.

[22]  Mahantapas Kundu,et al.  The journey of graph kernels through two decades , 2018, Comput. Sci. Rev..

[23]  Gerhard J. Woeginger,et al.  Graph Similarity and Approximate Isomorphism , 2018, MFCS.

[24]  Michalis Vazirgiannis,et al.  Kernel Graph Convolutional Neural Networks , 2017, ICANN.

[25]  Yixin Chen,et al.  An End-to-End Deep Learning Architecture for Graph Classification , 2018, AAAI.

[26]  Alessandro Sperduti,et al.  Measuring the expressivity of graph kernels through Statistical Learning Theory , 2017, Neurocomputing.

[27]  Kristian Kersting,et al.  Glocalized Weisfeiler-Lehman Graph Kernels: Global-Local Feature Maps of Graphs , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[28]  Yannis Stavrakas,et al.  Shortest-Path Graph Kernels for Document Similarity , 2017, EMNLP.

[29]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[30]  Regina Barzilay,et al.  Deriving Neural Architectures from Sequence and Graph Kernels , 2017, ICML.

[31]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[32]  Luc Brun,et al.  Chemoinformatics and stereoisomerism: A stereo graph kernel together with three new extensions , 2017, Pattern Recognit. Lett..

[33]  Kipton Barros,et al.  Learning molecular energies using localized graph kernels. , 2016, The Journal of chemical physics.

[34]  Michalis Vazirgiannis,et al.  Matching Node Embeddings for Graph Similarity , 2017, AAAI.

[35]  Razvan Pascanu,et al.  Interaction Networks for Learning about Objects, Relations and Physics , 2016, NIPS.

[36]  Alessandro Sperduti,et al.  Hyper-Parameter Tuning for Graph Kernels via Multiple Kernel Learning , 2016, ICONIP.

[37]  Daoqiang Zhang,et al.  Sub-network Based Kernels for Brain Network Classification , 2016, BCB.

[38]  Kristian Kersting,et al.  Faster Kernels for Graphs with Continuous Attributes via Hashing , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[39]  Francis wyffels,et al.  Automatic architectural style detection using one-class support vector machines and graph kernels , 2016 .

[40]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[41]  Xifeng Yan,et al.  A Fast Kernel for Attributed Graphs , 2016, SDM.

[42]  Roland Siegwart,et al.  Robust Visual Place Recognition with Graph Kernels , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Yang Liu,et al.  Contextual Weisfeiler-Lehman graph kernel for malware detection , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[44]  Martin Schäf,et al.  Detecting Similar Programs via The Weisfeiler-Leman Graph Kernel , 2016, ICSR.

[45]  Nils M. Kriege,et al.  On Valid Optimal Assignment Kernels and Applications to Graph Classification , 2016, NIPS.

[46]  James M. Bieman,et al.  Predicting metamorphic relations for testing scientific software: a machine learning approach using graph kernels , 2016, Softw. Test. Verification Reliab..

[47]  Vijay S. Pande,et al.  Molecular graph convolutions: moving beyond fingerprints , 2016, Journal of Computer-Aided Molecular Design.

[48]  Risi Kondor,et al.  The Multiscale Laplacian Graph Kernel , 2016, NIPS.

[49]  Akihiro Inokuchi,et al.  Hadamard Code Graph Kernels for Classifying Graphs , 2016, ICPRAM.

[50]  Meng Li,et al.  3D human motion retrieval using graph kernels based on adaptive graph construction , 2016, Comput. Graph..

[51]  Matthew B. Blaschko,et al.  The pyramid quantized Weisfeiler-Lehman graph representation , 2016, Neurocomputing.

[52]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[53]  Roman Garnett,et al.  Propagation kernels: efficient graph kernels from propagated information , 2015, Machine Learning.

[54]  S. V. N. Vishwanathan,et al.  A Structural Smoothing Framework For Robust Graph Comparison , 2015, NIPS.

[55]  Karsten M. Borgwardt,et al.  Halting in Random Walk Kernels , 2015, NIPS.

[56]  Alessandro Sperduti,et al.  Multiple Graph-Kernel Learning , 2015, 2015 IEEE Symposium Series on Computational Intelligence.

[57]  Steven de Rooij,et al.  Substructure counting graph kernels for machine learning from RDF data , 2015, J. Web Semant..

[58]  Andrea Torsello,et al.  Transitive Assignment Kernels for Structural Classification , 2015, SIMBAD.

[59]  Jan Ramon,et al.  Predicting Protein Function and Protein-Ligand Interaction with the 3D Neighborhood Kernel , 2015, Discovery Science.

[60]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[61]  Pinar Yanardag,et al.  Deep Graph Kernels , 2015, KDD.

[62]  Edwin R. Hancock,et al.  A Graph Kernel Based on the Jensen-Shannon Representation Alignment , 2015, IJCAI.

[63]  Luc De Raedt,et al.  Graph Invariant Kernels , 2015, IJCAI.

[64]  Edwin R. Hancock,et al.  An Aligned Subtree Kernel for Weighted Graphs , 2015, ICML.

[65]  Kristian Kersting,et al.  Explicit Versus Implicit Graph Feature Maps: A Computational Phase Transition for Walk Kernels , 2014, 2014 IEEE International Conference on Data Mining.

[66]  Bertrand Thirion,et al.  Graph-Based Inter-Subject Pattern Analysis of fMRI Data , 2014, PloS one.

[67]  Daoqiang Zhang,et al.  Topological graph kernel on multiple thresholded functional connectivity networks for mild cognitive impairment classification , 2014, Human brain mapping.

[68]  Devdatt P. Dubhashi,et al.  Global graph kernels using geometric embeddings , 2014, ICML.

[69]  Sandro Vega-Pons,et al.  Classification of inter-subject fMRI data based on graph kernels , 2014, 2014 International Workshop on Pattern Recognition in Neuroimaging.

[70]  Juan E. Tapiador,et al.  Evolution, Detection and Analysis of Malware for Smart Devices , 2014, IEEE Communications Surveys & Tutorials.

[71]  Yann LeCun,et al.  Spectral Networks and Deep Locally Connected Networks on Graphs , 2014 .

[72]  Marleen de Bruijne,et al.  Scalable kernels for graphs with continuous attributes , 2013, NIPS.

[73]  Hichem Sahbi,et al.  Directed Acyclic Graph Kernels for Action Recognition , 2013, 2013 IEEE International Conference on Computer Vision.

[74]  Konrad Rieck,et al.  Structural detection of android malware using embedded call graphs , 2013, AISec.

[75]  Devdatt P. Dubhashi,et al.  Entity disambiguation in anonymized graphs using graph kernels , 2013, CIKM.

[76]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[77]  Min Song,et al.  Text Categorization of Biomedical Data Sets Using Graph Kernels and a Controlled Vocabulary , 2013, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[78]  Roman Garnett,et al.  Graph Kernels for Object Category Prediction in Task-Dependent Robot Grasping , 2013, MLG 2013.

[79]  Sandro Vega-Pons,et al.  Brain Decoding via Graph Kernels , 2013, 2013 International Workshop on Pattern Recognition in Neuroimaging.

[80]  Rolf Backofen,et al.  A graph kernel approach for alignment-free domain–peptide interaction prediction with an application to human SH3 domains , 2013, Bioinform..

[81]  Xiao Liu,et al.  Fast multi-view segment graph kernel for object classification , 2013, Signal Process..

[82]  Gholam-Ali Hossein-Zadeh,et al.  Decoding brain states using backward edge elimination and graph kernels in fMRI connectivity networks , 2013, Journal of Neuroscience Methods.

[83]  Devdatt P. Dubhashi,et al.  Lovász ϑ function, SVMs and finding dense subgraphs , 2013, J. Mach. Learn. Res..

[84]  Jan Snajder,et al.  Recognizing Identical Events with Graph Kernels , 2013, ACL.

[85]  Chengqi Zhang,et al.  Nested Subtree Hash Kernels for Large-Scale Graph Classification over Streams , 2012, 2012 IEEE 12th International Conference on Data Mining.

[86]  Luc De Raedt,et al.  A Relational Kernel-Based Framework for Hierarchical Image Understanding , 2012, SSPR/SPR.

[87]  Nils M. Kriege,et al.  Subgraph Matching Kernels for Attributed Graphs , 2012, ICML.

[88]  Stephan Bloehdorn,et al.  Graph Kernels for RDF Data , 2012, ESWC.

[89]  G. Hossein-Zadeh,et al.  A method based on the granger causality and graph kernels for discriminating resting state from attentional task , 2012, 2012 International Conference on Biomedical Engineering (ICoBE).

[90]  Alessandro Sperduti,et al.  A Tree-Based Kernel for Graphs , 2012, SDM.

[91]  Curtis B. Storlie,et al.  Graph-based malware detection using dynamic analysis , 2011, Journal in Computer Virology.

[92]  Roberto Basili,et al.  Structured Lexical Similarity via Convolution Kernels on Dependency Trees , 2011, EMNLP.

[93]  Vladimir Batagelj,et al.  Fast algorithms for determining (generalized) core groups in social networks , 2011, Adv. Data Anal. Classif..

[94]  Pat Hanrahan,et al.  Characterizing structural relationships in scenes using graph kernels , 2011, ACM Trans. Graph..

[95]  Benoit Gaüzère,et al.  Two New Graph Kernels and Applications to Chemoinformatics , 2011, GbRPR.

[96]  Mario Vento,et al.  People Re-identification by Graph Kernels Methods , 2011, GbRPR.

[97]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[98]  Kurt Mehlhorn,et al.  Weisfeiler-Lehman Graph Kernels , 2011, J. Mach. Learn. Res..

[99]  T. Akutsu,et al.  Compound analysis via graph kernels incorporating chirality. , 2010, Journal of bioinformatics and computational biology.

[100]  Katja Filippova,et al.  Multi-Sentence Compression: Finding Shortest Paths in Word Graphs , 2010, COLING.

[101]  Luc Brun,et al.  Object classification based on graph kernels , 2010, 2010 International Conference on High Performance Computing & Simulation.

[102]  Giancarlo Mauri,et al.  An application of kernel methods to gene cluster temporal meta-analysis , 2010, Comput. Oper. Res..

[103]  Fabrizio Costa,et al.  Fast Neighborhood Subgraph Pairwise Distance Kernel , 2010, ICML.

[104]  Gisbert Schneider,et al.  Graph Kernels for Molecular Similarity , 2010, Molecular informatics.

[105]  Hisashi Kashima,et al.  A Linear-Time Graph Kernel , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[106]  Radu State,et al.  Malware analysis with graph kernels and support vector machines , 2009, 2009 4th International Conference on Malicious and Unwanted Software (MALWARE).

[107]  Jun Huan,et al.  Graph Wavelet Alignment Kernels for Drug Virtual Screening , 2009, J. Bioinform. Comput. Biol..

[108]  Kurt Mehlhorn,et al.  Efficient graphlet kernels for large graph comparison , 2009, AISTATS.

[109]  Alessio Micheli,et al.  Neural Network for Graphs: A Contextual Constructive Approach , 2009, IEEE Transactions on Neural Networks.

[110]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[111]  Kaspar Riesen,et al.  IAM Graph Database Repository for Graph Based Pattern Recognition and Machine Learning , 2008, SSPR/SPR.

[112]  Jari Björne,et al.  All-paths graph kernel for protein-protein interaction extraction with evaluation of cross-corpus learning , 2008, BMC Bioinformatics.

[113]  Kiyoshi Asai,et al.  Directed acyclic graph kernels for structural RNA analysis , 2008, BMC Bioinformatics.

[114]  Jari Björne,et al.  A Graph Kernel for Protein-Protein Interaction Extraction , 2008, BioNLP.

[115]  Roberto Basili,et al.  Tree Kernels for Semantic Role Labeling , 2008, CL.

[116]  Jean-Philippe Vert,et al.  The optimal assignment kernel is not positive definite , 2008, ArXiv.

[117]  Francis R. Bach,et al.  Graph kernels between point clouds , 2007, ICML '08.

[118]  Jean-Philippe Vert,et al.  Graph kernels based on tree patterns for molecules , 2006, Machine Learning.

[119]  S. V. N. Vishwanathan,et al.  Graph kernels , 2007 .

[120]  Peter Geibel,et al.  Learning Models of Relational MDPs Using Graph Kernels , 2007, MICAI.

[121]  Alessio Ceroni,et al.  Classification of small molecules by two- and three-dimensional decomposition kernels , 2007, Bioinform..

[122]  Zaïd Harchaoui,et al.  Image Classification with Segmentation Graph Kernels , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[123]  Trevor Darrell,et al.  The Pyramid Match Kernel: Efficient Learning with Sets of Features , 2007, J. Mach. Learn. Res..

[124]  Natasa Przulj,et al.  Biological network comparison using graphlet degree distribution , 2007, Bioinform..

[125]  Hans-Peter Kriegel,et al.  Graph Kernels For Disease Outcome Prediction From Protein-Protein Interaction Networks , 2006, Pacific Symposium on Biocomputing.

[126]  George Karypis,et al.  Comparison of descriptor spaces for chemical compound retrieval and classification , 2006, Sixth International Conference on Data Mining (ICDM'06).

[127]  Alessandro Moschitti,et al.  Efficient Convolution Kernels for Dependency and Constituent Syntactic Trees , 2006, ECML.

[128]  Thomas Gärtner,et al.  Graph kernels and Gaussian processes for relational reinforcement learning , 2006, Machine Learning.

[129]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[130]  Alessandro Moschitti,et al.  Making Tree Kernels Practical for Natural Language Learning , 2006, EACL.

[131]  Alessandro Vespignani,et al.  Large scale networks fingerprinting and visualization using the k-core decomposition , 2005, NIPS.

[132]  Hans-Peter Kriegel,et al.  Shortest-path kernels on graphs , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[133]  Razvan C. Bunescu,et al.  A Shortest Path Dependency Kernel for Relation Extraction , 2005, HLT.

[134]  Pierre Baldi,et al.  Graph kernels for chemical informatics , 2005, Neural Networks.

[135]  Andreas Zell,et al.  Optimal assignment kernels for attributed molecular graphs , 2005, ICML.

[136]  Tatsuya Akutsu,et al.  Graph Kernels for Molecular Structure-Activity Relationship Analysis with Support Vector Machines , 2005, J. Chem. Inf. Model..

[137]  Pierre Baldi,et al.  Kernels for small molecules and the prediction of mutagenicity, toxicity and anti-cancer activity , 2005, ISMB.

[138]  Hans-Peter Kriegel,et al.  Protein function prediction via graph kernels , 2005, ISMB.

[139]  Thomas Gärtner,et al.  Cyclic pattern kernels for predictive graph mining , 2004, KDD.

[140]  Inderjit S. Dhillon,et al.  Kernel k-means: spectral clustering and normalized cuts , 2004, KDD.

[141]  Aron Culotta,et al.  Dependency Tree Kernels for Relation Extraction , 2004, ACL.

[142]  Tatsuya Akutsu,et al.  Extensions of marginalized graph kernels , 2004, ICML.

[143]  Rada Mihalcea,et al.  TextRank: Bringing Order into Text , 2004, EMNLP.

[144]  Mario Vento,et al.  Thirty Years Of Graph Matching In Pattern Recognition , 2004, Int. J. Pattern Recognit. Artif. Intell..

[145]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[146]  Francesca Odone,et al.  Histogram intersection kernel for image classification , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[147]  Jeffrey J. Sutherland,et al.  Spline-Fitting with a Genetic Algorithm: A Method for Developing Classification Structure-Activity Relationships , 2003, J. Chem. Inf. Comput. Sci..

[148]  Hisashi Kashima,et al.  Marginalized Kernels Between Labeled Graphs , 2003, ICML.

[149]  P. Dobson,et al.  Distinguishing enzyme structures from non-enzymes without alignments. , 2003, Journal of molecular biology.

[150]  Thomas Gärtner,et al.  A survey of kernels for structured data , 2003, SKDD.

[151]  Dmitry Zelenko,et al.  Kernel Methods for Relation Extraction , 2002, J. Mach. Learn. Res..

[152]  Thomas Gärtner,et al.  On Graph Kernels: Hardness Results and Efficient Alternatives , 2003, COLT.

[153]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[154]  E. Ordentlich,et al.  Inequalities for the L1 Deviation of the Empirical Distribution , 2003 .

[155]  Jan Ramon,et al.  Expressivity versus efficiency of graph kernels , 2003 .

[156]  Alexander J. Smola,et al.  Kernels and Regularization on Graphs , 2003, COLT.

[157]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[158]  John D. Lafferty,et al.  Diffusion Kernels on Graphs and Other Discrete Input Spaces , 2002, ICML.

[159]  Jean-Philippe Vert,et al.  A tree kernel to analyse phylogenetic profiles , 2002, ISMB.

[160]  Alexander J. Smola,et al.  Fast Kernels for String and Tree Matching , 2002, NIPS.

[161]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[162]  Michael Collins,et al.  Convolution Kernels for Natural Language , 2001, NIPS.

[163]  Hannu Toivonen,et al.  Statistical evaluation of the predictive toxicology challenge , 2000 .

[164]  Christopher K. I. Williams,et al.  Using the Nyström Method to Speed Up Kernel Machines , 2000, NIPS.

[165]  David Haussler,et al.  Convolution kernels on discrete structures , 1999 .

[166]  Klaus Obermayer,et al.  Classification on Pairwise Proximity Data , 1998, NIPS.

[167]  Alessandro Sperduti,et al.  Supervised neural networks for the classification of structures , 1997, IEEE Trans. Neural Networks.

[168]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[169]  A. Debnath,et al.  Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. Correlation with molecular orbital energies and hydrophobicity. , 1991, Journal of medicinal chemistry.

[170]  Stephen B. Seidman,et al.  Network structure and minimum degree , 1983 .

[171]  Leland L. Beck,et al.  Smallest-last ordering and clustering and graph coloring algorithms , 1983, JACM.

[172]  László Lovász,et al.  On the Shannon capacity of a graph , 1979, IEEE Trans. Inf. Theory.

[173]  Robert L. Hemminger,et al.  Graph reconstruction - a survey , 1977, J. Graph Theory.

[174]  G. Levi.  A note on the derivation of maximal common subgraphs of two directed or undirected graphs , 1973 .

[175]  J. Gower.  A General Coefficient of Similarity and Some of Its Properties , 1971 .

[176]  M. Aizerman,et al.  Theoretical Foundations of the Potential Function Method in Pattern Recognition Learning , 1964 .

[177]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .