Product Graph-based Higher Order Contextual Similarities for Inexact Subgraph Matching

Many algorithms formulate graph matching as an optimization of an objective function of pairwise quantification of nodes and edges of two graphs to be matched. Pairwise measurements usually consider local attributes but disregard contextual information involved in graph structures. We address this issue by proposing contextual similarities between pairs of nodes. This is done by considering the tensor product graph (TPG) of two graphs to be matched, where each node is an ordered pair of nodes of the operand graphs. Contextual similarities between a pair of nodes are computed by accumulating weighted walks (normalized pairwise similarities) terminating at the corresponding paired node in TPG. Once the contextual similarities are obtained, we formulate subgraph matching as a node and edge selection problem in TPG. We use contextual similarities to construct an objective function and optimize it with a linear programming approach. Since random walk formulation through TPG takes into account higher order information, it is not a surprise that we obtain more reliable similarities and better discrimination among the nodes and edges. Experimental results shown on synthetic as well as real benchmarks illustrate that higher order contextual similarities add discriminating power and allow one to find approximate solutions to the subgraph matching problem.

[1]  S. V. N. Vishwanathan,et al.  Graph kernels , 2007 .

[2]  Amnon Shashua,et al.  Probabilistic graph and hypergraph matching , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Martial Hebert,et al.  An Integer Projected Fixed Point Method for Graph Matching and MAP Inference , 2009, NIPS.

[4]  Martial Hebert,et al.  Unsupervised Learning for Graph Matching , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[6]  Umapada Pal,et al.  A Product Graph Based Method for Dual Subgraph Matching Applied to Symbol Spotting , 2013, GREC.

[7]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[8]  Jean Ponce,et al.  A Tensor-Based Algorithm for High-Order Graph Matching , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  J. Munkres ALGORITHMS FOR THE ASSIGNMENT AND TRANSIORTATION tROBLEMS* , 1957 .

[10]  Julian R. Ullmann,et al.  An Algorithm for Subgraph Isomorphism , 1976, J. ACM.

[11]  Vladimir Kolmogorov,et al.  Feature Correspondence Via Graph Matching: Models and Global Optimization , 2008, ECCV.

[12]  Hong Qiao,et al.  A Convex-Concave Relaxation Procedure Based Subgraph Matching Algorithm , 2012, ACML.

[13]  Mario Vento,et al.  Thirty Years Of Graph Matching In Pattern Recognition , 2004, Int. J. Pattern Recognit. Artif. Intell..

[14]  Lorenzo Livi,et al.  Parallel algorithms for tensor product-based inexact graph matching , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[15]  Tony P. Pridmore,et al.  Building Synthetic Graphical Documents for Performance Evaluation , 2007, GREC.

[16]  Tommi S. Jaakkola,et al.  Partially labeled classification with Markov random walks , 2001, NIPS.

[17]  Shinji Umeyama,et al.  An Eigendecomposition Approach to Weighted Graph Matching Problems , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Prasad Raghavendra,et al.  Optimal algorithms and inapproximability results for every CSP? , 2008, STOC.

[19]  Horst Bunke,et al.  On a relation between graph edit distance and maximum common subgraph , 1997, Pattern Recognit. Lett..

[20]  Philip H. S. Torr,et al.  Solving Markov Random Fields using Semi Definite Programming , 2003, AISTATS.

[21]  Jean-Yves Ramel,et al.  Fuzzy multilevel graph embedding , 2013, Pattern Recognit..

[22]  Junzhou Huang,et al.  Optimal object matching via convexification and composition , 2011, 2011 International Conference on Computer Vision.

[23]  Vladimir Kolmogorov,et al.  A Dual Decomposition Approach to Feature Correspondence , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Jianbo Shi,et al.  Balanced Graph Matching , 2006, NIPS.

[25]  Jean Ponce,et al.  A graph-matching kernel for object categorization , 2011, 2011 International Conference on Computer Vision.

[26]  William Brendel,et al.  Learning spatiotemporal graphs of human activities , 2011, 2011 International Conference on Computer Vision.

[27]  Horst Bunke,et al.  Error Correcting Graph Matching: On the Influence of the Underlying Cost Function , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Thomas Gärtner,et al.  On Graph Kernels: Hardness Results and Efficient Alternatives , 2003, COLT.

[29]  Volkmar Frinken,et al.  A Fast Matching Algorithm for Graph-Based Handwriting Recognition , 2013, GbRPR.

[30]  Terry Caelli,et al.  An eigenspace projection clustering method for inexact graph matching , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Barend J. van Wyk,et al.  A POCS-Based Graph Matching Algorithm , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Salih O. Duffuaa,et al.  A Linear Programming Approach for the Weighted Graph Matching Problem , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Hong Qiao,et al.  Partial correspondence based on subgraph matching , 2013, Neurocomputing.

[34]  Lei Wang,et al.  Improving Graph Matching via Density Maximization , 2013, 2013 IEEE International Conference on Computer Vision.

[35]  Edwin R. Hancock,et al.  Spectral correspondence for point pattern matching , 2003, Pattern Recognit..

[36]  Steven Gold,et al.  A Graduated Assignment Algorithm for Graph Matching , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Wenyin Liu,et al.  An interactive example-driven approach to graphics recognition in engineering drawings , 2006, International Journal of Document Analysis and Recognition (IJDAR).

[38]  Umapada Pal,et al.  Near Convex Region Adjacency Graph and Approximate Neighborhood String Matching for Symbol Spotting in Graphical Documents , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[39]  Hong Qiao,et al.  Feature correspondence based on directed structural model matching , 2015, Image Vis. Comput..

[40]  Mario Vento,et al.  A (sub)graph isomorphism algorithm for matching large graphs , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Regina Tyshkevich,et al.  Graph isomorphism problem , 1985 .

[42]  A. Terras,et al.  Zeta functions of finite graphs and coverings, III , 1996 .

[43]  Jean Ponce,et al.  Learning Graphs to Match , 2013, 2013 IEEE International Conference on Computer Vision.

[44]  Minsu Cho,et al.  Reweighted Random Walks for Graph Matching , 2010, ECCV.

[45]  Christoph Schnörr,et al.  Probabilistic Subgraph Matching Based on Convex Relaxation , 2005, EMMCVPR.

[46]  Xu Yang,et al.  Adaptive Graph Matching , 2018, IEEE Transactions on Cybernetics.

[47]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[48]  Minsu Cho,et al.  Mode-seeking on graphs via random walks , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Xu Yang,et al.  Incorporating Discrete Constraints Into Random Walk-Based Graph Matching , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[50]  Hong Qiao,et al.  Point correspondence by a new third order graph matching algorithm , 2017, Pattern Recognit..

[51]  King-Sun Fu,et al.  Subgraph error-correcting isomorphisms for syntactic pattern recognition , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[52]  Mario Vento,et al.  A long trip in the charming world of graphs for Pattern Recognition , 2015, Pattern Recognit..

[53]  Fernando De la Torre,et al.  Deformable Graph Matching , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Alexei A. Efros,et al.  Discovering Texture Regularity as a Higher-Order Correspondence Problem , 2006, ECCV.

[55]  Longin Jan Latecki,et al.  Affinity Learning with Diffusion on Tensor Product Graph , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Hong Qiao,et al.  An Extended Path Following Algorithm for Graph-Matching Problem , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[57]  Stella X. Yu,et al.  Linear Scale and Rotation Invariant Matching , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Minsu Cho,et al.  Hyper-graph matching via reweighted random walks , 2011, CVPR 2011.

[59]  King-Sun Fu,et al.  Error-Correcting Isomorphisms of Attributed Relational Graphs for Pattern Analysis , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[60]  Martial Hebert,et al.  A spectral technique for correspondence problems using pairwise constraints , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[61]  Alexander J. Smola,et al.  Learning Graph Matching , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Horst Bunke,et al.  Bridging the Gap between Graph Edit Distance and Kernel Machines , 2007, Series in Machine Perception and Artificial Intelligence.

[63]  A. Terras,et al.  Zeta Functions of Finite Graphs and Coverings , 1996 .

[64]  Edwin R. Hancock,et al.  Backtrackless Walks on a Graph , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[65]  Kaspar Riesen,et al.  Towards the unification of structural and statistical pattern recognition , 2012, Pattern Recognit. Lett..

[66]  Jean-Yves Ramel,et al.  Spotting Symbols in Line Drawing Images Using Graph Representations , 2007, GREC.

[67]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[68]  Fernando De la Torre,et al.  Factorized Graph Matching , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  M. Zaslavskiy,et al.  A Path Following Algorithm for the Graph Matching Problem , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[70]  Mario Vento,et al.  Graph Matching and Learning in Pattern Recognition in the Last 10 Years , 2014, Int. J. Pattern Recognit. Artif. Intell..

[71]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[72]  Josep Lladós,et al.  Symbol Recognition by Error-Tolerant Subgraph Matching between Region Adjacency Graphs , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[73]  Yves Lecourtier,et al.  An integer linear program for substitution-tolerant subgraph isomorphism and its use for symbol spotting in technical drawings , 2012, Pattern Recognit..

[74]  Josep Lladós,et al.  A performance evaluation protocol for symbol spotting systems in terms of recognition and location indices , 2009, International Journal on Document Analysis and Recognition (IJDAR).

[75]  P. Foggia,et al.  Performance evaluation of the VF graph matching algorithm , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[76]  Umapada Pal,et al.  A symbol spotting approach in graphical documents by hashing serialized graphs , 2013, Pattern Recognit..

[77]  Kaleem Siddiqi,et al.  Matching Hierarchical Structures Using Association Graphs , 1998, IEEE Trans. Pattern Anal. Mach. Intell..