Training Free Graph Neural Networks for Graph Matching

We present TFGM (Training Free Graph Matching), a framework to boost the performance of Graph Neural Networks (GNNs) based graph matching without training. TFGM sidesteps two crucial problems when training GNNs: 1) the limited supervision due to expensive annotation, and 2) training’s computational cost. A basic framework, BasicTFGM, is first proposed by adopting the inference stage of graph matching methods. Our analysis shows that the BasicTFGM is a linear relaxation to the quadratic assignment formulation of graph matching. This guarantees the preservation of structure compatibility and an efficient polynomial complexity. Empirically, we further improve the BasicTFGM by handcrafting two types of matching priors into the architecture of GNNs: comparing node neighborhoods of different localities and utilizing annotation data if available. For evaluation, we conduct extensive experiments on a broad set of settings, including supervised keypoint matching between images, semi-supervised entity alignment between knowledge graphs, and unsupervised alignment between protein interaction networks. Applying TFGM on various GNNs shows promising improvements over baselines. Further ablation studies demonstrate the effective and efficient training-free property of TFGM. Our code is available at https://github.com/acharkq/TrainingFree-Graph-Matching.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Yong Cao,et al.  CoLink: An Unsupervised Framework for User Identity Linkage , 2018, AAAI.

[3]  Lu Yu,et al.  Semi-Supervised Entity Alignment via Knowledge Graph Embedding with Awareness of Degree Difference , 2019, WWW.

[4]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[5]  Christos Faloutsos,et al.  Collective Multi-type Entity Alignment Between Knowledge Graphs , 2020, WWW.

[6]  Robert Patro,et al.  Global network alignment using multiscale spectral signatures , 2012, Bioinform..

[7]  Man Lan,et al.  Boosting the Speed of Entity Alignment 10 ×: Dual Attention Matching Network with Normalized Hard Sample Mining , 2021, WWW.

[8]  Yizhou Sun,et al.  Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks , 2019, NeurIPS.

[9]  Joan Bruna,et al.  On Graph Neural Networks versus Graph-Augmented MLPs , 2021, ICLR.

[10]  Jingping Bi,et al.  Cross-Network Embedding for Multi-Network Alignment , 2019, WWW.

[11]  Ruoming Jin,et al.  Robust Network Alignment via Attack Signal Scaling and Adversarial Perturbation Elimination , 2021, WWW.

[12]  Yue Wang,et al.  Deep Closest Point: Learning Representations for Point Cloud Registration , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[13]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[14]  Philip S. Yu,et al.  Multiple Anonymized Social Networks Alignment , 2015, 2015 IEEE International Conference on Data Mining.

[15]  Anton van den Hengel,et al.  Is margin preserved after random projection? , 2012, ICML.

[16]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[17]  Nigel Collier,et al.  Visual Pivoting for (Unsupervised) Entity Alignment , 2020, AAAI.

[18]  Tijana Milenkovic,et al.  MAGNA++: Maximizing Accuracy in Global Network Alignment via both node and edge conservation , 2015, Bioinform..

[19]  Zhiyuan Liu,et al.  Exploring and Evaluating Attributes, Values, and Structure for Entity Alignment , 2020, EMNLP.

[20]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[21]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Bing Xue,et al.  KerGM: Kernelized Graph Matching , 2019, NeurIPS.

[23]  Chengkai Li,et al.  A benchmarking study of embedding-based entity alignment for knowledge graphs , 2020, Proc. VLDB Endow..

[24]  KingsfordCarl,et al.  Global network alignment using multiscale spectral signatures , 2012 .

[25]  Heinrich Müller,et al.  SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Hanghang Tong,et al.  BRIGHT: A Bridging Algorithm for Network Alignment , 2021, WWW.

[27]  Jonghun Park,et al.  Image-to-Image Retrieval by Learning Similarity between Scene Graphs , 2020, AAAI.

[28]  Yann LeCun,et al.  What is the best multi-stage architecture for object recognition? , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[29]  Davide Eynard,et al.  SIGN: Scalable Inception Graph Neural Networks , 2020, ArXiv.

[30]  Jiuyang Tang,et al.  An Experimental Study of State-of-the-Art Entity Alignment Approaches , 2020, IEEE Transactions on Knowledge and Data Engineering.

[31]  Lei Meng,et al.  The post-genomic era of biological network alignment , 2015, EURASIP J. Bioinform. Syst. Biol..

[32]  Cristian Sminchisescu,et al.  Deep Learning of Graph Matching , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  Tomás Pajdla,et al.  Neighbourhood Consensus Networks , 2018, NeurIPS.

[34]  Jasper Snoek,et al.  Sinkhorn Networks: Using Optimal Transport Techniques to Learn Permutations , 2017 .

[35]  Georg Martius,et al.  Deep Graph Matching via Blackbox Differentiation of Combinatorial Solvers , 2020, ECCV.

[36]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[37]  Hwann-Tzong Chen,et al.  Multi-object tracking using dynamical graph matching , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[38]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[39]  Jean Ponce,et al.  Learning Graphs to Match , 2013, 2013 IEEE International Conference on Computer Vision.

[40]  Christian Theobalt,et al.  A Convex Relaxation for Multi-Graph Matching , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Mario Vento,et al.  Thirty Years Of Graph Matching In Pattern Recognition , 2004, Int. J. Pattern Recognit. Artif. Intell..

[42]  F. Feltus,et al.  Gene Coexpression Network Alignment and Conservation of Gene Modules between Two Grass Species: Maize and Rice[C][W][OA] , 2011, Plant Physiology.

[43]  Jean-Loup Guillaume,et al.  Fast unfolding of communities in large networks , 2008, 0803.0476.

[44]  Carlo Zaniolo,et al.  Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment , 2016, IJCAI.

[45]  Jugal K. Kalita,et al.  Global Alignment of Protein-Protein Interaction Networks: A Survey , 2016, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[46]  Chunxiao Liu,et al.  Graph Structured Network for Image-Text Matching , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Alexander J. Smola,et al.  Learning Graph Matching , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Zhenghao Chen,et al.  On Random Weights and Unsupervised Feature Learning , 2011, ICML.

[49]  Tommy W. S. Chow,et al.  Comments on "Stochastic choice of basis functions in adaptive function approximation and the functional-link net" [and reply] , 1997, IEEE Trans. Neural Networks.

[50]  Sean R. Collins,et al.  Toward a Comprehensive Atlas of the Physical Interactome of Saccharomyces cerevisiae*S , 2007, Molecular & Cellular Proteomics.

[51]  Carsten Rother,et al.  A Study of Lagrangean Decompositions and Dual Ascent Solvers for Graph Matching , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  Yansong Feng,et al.  Neighborhood Matching Network for Entity Alignment , 2020, ACL.

[53]  Lorenzo Torresani,et al.  Detect-and-Track: Efficient Pose Estimation in Videos , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[54]  Zhichun Wang,et al.  Cross-lingual Knowledge Graph Alignment via Graph Convolutional Networks , 2018, EMNLP.

[55]  Arnab Bhattacharya,et al.  Neighbor-Aware Search for Approximate Labeled Graph Matching using the Chi-Square Statistics , 2017, WWW.

[56]  Junchi Yan,et al.  Learning Combinatorial Embedding Networks for Deep Graph Matching , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[57]  Tomasz Malisiewicz,et al.  SuperGlue: Learning Feature Matching With Graph Neural Networks , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[59]  Yoh-Han Pao,et al.  Stochastic choice of basis functions in adaptive function approximation and the functional-link net , 1995, IEEE Trans. Neural Networks.

[60]  Si Zhang,et al.  Multilevel Network Alignment , 2019, WWW.

[61]  Jitendra Malik,et al.  Poselets: Body part detectors trained using 3D human pose annotations , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[62]  Nils M. Kriege,et al.  Deep Graph Matching Consensus , 2020, ICLR.

[63]  Junchi Yan,et al.  Neural Graph Matching Network: Learning Lawler’s Quadratic Assignment Problem With Extension to Hypergraph and Multiple-Graph Matching , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  Anupam Gupta,et al.  An elementary proof of the Johnson-Lindenstrauss Lemma , 1999 .

[65]  Iryna Gurevych,et al.  Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks , 2019, EMNLP.

[66]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[67]  Tijana Milenkovic,et al.  MAGNA: Maximizing Accuracy in Global Network Alignment , 2013, Bioinform..

[68]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[69]  Yizhou Sun,et al.  Are Powerful Graph Neural Nets Necessary? A Dissection on Graph Classification , 2019, ArXiv.

[70]  T. Milenković,et al.  Systems biology MAGNA 11 : Maximizing Accuracy in Global Network Alignment via both node and edge conservation , 2015 .

[71]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[72]  Volker Tresp,et al.  Active Learning for Entity Alignment , 2020, ECIR.

[73]  Silvio Savarese,et al.  Universal Correspondence Network , 2016, NIPS.

[74]  Wei Hu,et al.  Cross-Lingual Entity Alignment via Joint Attribute-Preserving Embedding , 2017, SEMWEB.

[75]  Yansong Feng,et al.  Cross-lingual Knowledge Graph Alignment via Graph Matching Neural Network , 2019, ACL.

[76]  Hanghang Tong,et al.  Attent: Active Attributed Network Alignment , 2021, WWW.

[77]  Yong Yu,et al.  Conceptual Graph Matching for Semantic Search , 2002, ICCS.

[78]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[79]  Alex X. Liu,et al.  Deep Graph Matching and Searching for Semantic Code Retrieval , 2020, ArXiv.

[80]  Shahin Mohammadi,et al.  Low Rank Spectral Network Alignment , 2018, WWW.

[81]  Takanori Maehara,et al.  Revisiting Graph Neural Networks: All We Have is Low-Pass Filters , 2019, ArXiv.

[82]  Samy Bengio,et al.  Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks , 2019, KDD.