Hierarchical stochastic graphlet embedding for graph-based pattern recognition

Despite being very successful within the pattern recognition and machine learning community, graph-based methods are often unusable because of the lack of mathematical operations defined in graph domain. Graph embedding, which maps graphs to a vectorial space, has been proposed as a way to tackle these difficulties enabling the use of standard machine learning techniques. However, it is well known that graph embedding functions usually suffer from the loss of structural information. In this paper, we consider the hierarchical structure of a graph as a way to mitigate this loss of information. The hierarchical structure is constructed by topologically clustering the graph nodes and considering each cluster as a node in the upper hierarchical level. Once this hierarchical structure is constructed, we consider several configurations to define the mapping into a vector space given a classical graph embedding, in particular, we propose to make use of the stochastic graphlet embedding (SGE). Broadly speaking, SGE produces a distribution of uniformly sampled low-to-high-order graphlets as a way to embed graphs into the vector space. In what follows, the coarse-to-fine structure of a graph hierarchy and the statistics fetched by the SGE complements each other and includes important structural information with varied contexts. Altogether, these two techniques substantially cope with the usual information loss involved in graph embedding techniques, obtaining a more robust graph representation. This fact has been corroborated through a detailed experimental evaluation on various benchmark graph datasets, where we outperform the state-of-the-art methods.

[1]  Thomas Gärtner,et al.  A survey of kernels for structured data , 2003, SKDD.

[2]  Kaspar Riesen,et al.  Improving vector space embedding of graphs through feature selection algorithms , 2011, Pattern Recognit..

[3]  Robert P. W. Duin,et al.  The Dissimilarity Representation for Pattern Recognition - Foundations and Applications , 2005, Series in Machine Perception and Artificial Intelligence.

[4]  Juan Carlos Niebles,et al.  A Hierarchical Model of Shape and Appearance for Human Action Classification , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Karsten M. Borgwardt,et al.  The graphlet spectrum , 2009, ICML '09.

[6]  Josep Lladós,et al.  Hierarchical Graph Representation for Symbol Spotting in Graphical Document Images , 2012, SSPR/SPR.

[7]  George Karypis,et al.  Frequent subgraph discovery , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[8]  E. Ordentlich,et al.  Inequalities for the L1 Deviation of the Empirical Distribution , 2003 .

[9]  Donald F. Towsley,et al.  Diffusion-Convolutional Neural Networks , 2015, NIPS.

[10]  Mohammed J. Zaki,et al.  Is There a Best Quality Metric for Graph Clusters? , 2011, ECML/PKDD.

[11]  Rudolf Kruse,et al.  of Neural Networks , 2008 .

[12]  Pinar Yanardag,et al.  Deep Graph Kernels , 2015, KDD.

[13]  P. Héroux,et al.  Frequent Graph Discovery: Application to Line Drawing Document Images , 2005 .

[14]  Joshua D. Schwartz,et al.  Hierarchical Matching of Deformable Shapes , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[16]  Ali Shokoufandeh,et al.  Indexing hierarchical structures using graph spectra , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Kaspar Riesen,et al.  Graph Classification by Means of Lipschitz Embedding , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[19]  Thomas Gärtner,et al.  Cyclic pattern kernels for predictive graph mining , 2004, KDD.

[20]  Natasa Przulj,et al.  Biological network comparison using graphlet degree distribution , 2007, Bioinform..

[21]  Kaspar Riesen,et al.  A Novel Graph Database for Handwritten Word Images , 2016, S+SSPR.

[22]  Hai Jin,et al.  Layered shape matching and registration: Stochastic sampling with hierarchical graph representation , 2008, 2008 19th International Conference on Pattern Recognition.

[23]  Shuai Li,et al.  Distributed Clustering of Linear Bandits in Peer to Peer Networks , 2016, ICML.

[24]  Shuai Li,et al.  Improved Algorithm on Online Clustering of Bandits , 2019, IJCAI.

[25]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[26]  Edwin R. Hancock,et al.  Backtrackless Walks on a Graph , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[28]  Hichem Sahbi,et al.  Stochastic Graphlet Embedding , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Zhi-Li Zhang,et al.  Hunt For The Unique, Stable, Sparse And Fast Feature Learning On Graphs , 2017, NIPS.

[30]  Kaspar Riesen,et al.  Bipartite Graph Matching for Computing the Edit Distance of Graphs , 2007, GbRPR.

[31]  Josep Lladós,et al.  Hierarchical Plausibility-Graphs for Symbol Spotting in Graphical Documents , 2013, GREC.

[32]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[33]  Abdolreza Mirzaei,et al.  Hierarchical graph embedding in vector space by graph pyramid , 2017, Pattern Recognit..

[34]  Francesc Comellas,et al.  Reconstruction of Networks from Their Betweenness Centrality , 2008, EvoWorkshops.

[35]  Mario Vento,et al.  Graph Matching and Learning in Pattern Recognition in the Last 10 Years , 2014, Int. J. Pattern Recognit. Artif. Intell..

[36]  Yves Lecourtier,et al.  An integer linear program for substitution-tolerant subgraph isomorphism and its use for symbol spotting in technical drawings , 2012, Pattern Recognit..

[37]  Horst Bunke,et al.  An Error-Tolerant Approximate Matching Algorithm for Attributed Planar Graphs and Its Application to Fingerprint Classification , 2004, SSPR/SPR.

[38]  S. V. N. Vishwanathan,et al.  Graph kernels , 2007 .

[39]  Salvatore Tabbone,et al.  Graph Embedding Using Constant Shift Embedding , 2010, ICPR Contests.

[40]  Horst Bunke,et al.  Bridging the Gap between Graph Edit Distance and Kernel Machines , 2007, Series in Machine Perception and Artificial Intelligence.

[41]  Kurt Mehlhorn,et al.  Efficient graphlet kernels for large graph comparison , 2009, AISTATS.

[42]  Leonard M. Freeman,et al.  A set of measures of centrality based upon betweenness , 1977 .

[43]  Shuai Li,et al.  Collaborative Filtering Bandits , 2015, SIGIR.

[44]  Azriel Rosenfeld,et al.  A Pyramid Framework for Early Vision: Multiresolutional Computer Vision , 1993 .

[45]  Kurt Mehlhorn,et al.  Graph Algorithm and NP-Completeness , 1984 .

[46]  Alicia Fornés,et al.  Pyramidal Stochastic Graphlet Embedding for Document Pattern Classification , 2017, 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR).

[47]  Luc Brun,et al.  Hierarchical Bag of Paths for Kernel Based Shape Classification , 2008, SSPR/SPR.

[48]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[49]  H. Kashima,et al.  Kernels for graphs , 2004 .

[50]  Christoph Schnörr,et al.  Probabilistic Subgraph Matching Based on Convex Relaxation , 2005, EMMCVPR.

[51]  Hichem Sahbi,et al.  High Order Stochastic Graphlet Embedding for Graph-Based Pattern Recognition , 2017, ArXiv.

[52]  Kamil Adamczewski,et al.  Subgraph matching using compactness prior for robust feature correspondence , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Nils M. Kriege,et al.  Subgraph Matching Kernels for Attributed Graphs , 2012, ICML.

[54]  Massimo Piccardi,et al.  Discriminative prototype selection methods for graph embedding , 2013, Pattern Recognit..

[55]  Mark E. J. Newman A measure of betweenness centrality based on random walks , 2005, Soc. Networks.

[56]  Ernest Valveny,et al.  Graph embedding in vector spaces by node attribute statistics , 2012, Pattern Recognit..

[57]  Markus Ulrich,et al.  Combining Scale-Space and Similarity-Based Aspect Graphs for Fast 3D Object Recognition , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Azriel Rosenfeld,et al.  A Pyramid Framework for Early Vision , 1994 .

[59]  Kaleem Siddiqi,et al.  Matching Hierarchical Structures Using Association Graphs , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[60]  Mario Vento,et al.  Thirty Years Of Graph Matching In Pattern Recognition , 2004, Int. J. Pattern Recognit. Artif. Intell..

[61]  Kaspar Riesen,et al.  IAM Graph Database Repository for Graph Based Pattern Recognition and Machine Learning , 2008, SSPR/SPR.

[62]  Edwin R. Hancock,et al.  Pattern Vectors from Algebraic Graph Theory , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[63]  Karsten M. Borgwardt,et al.  Fast subtree kernels on graphs , 2009, NIPS.

[64]  Alexander J. Smola,et al.  Kernels and Regularization on Graphs , 2003, COLT.

[65]  Hans-Peter Kriegel,et al.  Shortest-path kernels on graphs , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[66]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[67]  Jean-Yves Ramel,et al.  Fuzzy multilevel graph embedding , 2013, Pattern Recognit..

[68]  Francisco Sandoval Hernández,et al.  The Construction of Bounded Irregular Pyramids with a Union-Find Decimation Process , 2007, GbRPR.

[69]  Alberto Sanfeliu,et al.  Efficient algorithms for matching attributed graphs and function-described graphs , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[70]  Risi Kondor,et al.  The Multiscale Laplacian Graph Kernel , 2016, NIPS.

[71]  Umapada Pal,et al.  Product Graph-based Higher Order Contextual Similarities for Inexact Subgraph Matching , 2017, Pattern Recognit..

[72]  Christine Solnon,et al.  AllDifferent-based filtering for subgraph isomorphism , 2010, Artif. Intell..

[73]  Shuai Li,et al.  On Context-Dependent Clustering of Bandits , 2016, ICML.

[74]  Terry Caelli,et al.  An eigenspace projection clustering method for inexact graph matching , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[75]  Eric Saund A Graph Lattice Approach to Maintaining and Learning Dense Collections of Subgraphs as Image Features , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[76]  Karsten M. Borgwardt,et al.  The skew spectrum of graphs , 2008, ICML '08.

[77]  John D. Lafferty,et al.  Diffusion Kernels on Statistical Manifolds , 2005, J. Mach. Learn. Res..

[78]  Edwin R. Hancock,et al.  A Riemannian approach to graph embedding , 2007, Pattern Recognit..

[79]  Kaspar Riesen,et al.  Approximate graph edit distance computation by means of bipartite graph matching , 2009, Image Vis. Comput..

[80]  Alicia Fornés,et al.  Error-Tolerant Coarse-to-Fine Matching Model for Hierarchical Graphs , 2017, GbRPR.

[81]  Edward H. Adelson,et al.  PYRAMID METHODS IN IMAGE PROCESSING. , 1984 .

[82]  Mario Vento,et al.  A long trip in the charming world of graphs for Pattern Recognition , 2015, Pattern Recognit..

[83]  Aneesh Krishna,et al.  Robust Face Recognition by Utilizing Color Information and Sparse Representation , 2014, Int. J. Pattern Recognit. Artif. Intell..

[84]  Narendra Ahuja,et al.  From Region Based Image Representation to Object Discovery and Recognition , 2010, SSPR/SPR.

[85]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[86]  Kurt Mehlhorn,et al.  Weisfeiler-Lehman Graph Kernels , 2011, J. Mach. Learn. Res..