GAMnet: Robust Feature Matching via Graph Adversarial-Matching Network

Recently, deep graph matching (GM) methods have gained increasing attention. These methods integrate graph nodes¡¯s embedding, node/edges¡¯s affinity learning and final correspondence solver together in an end-to-end manner. For deep graph matching problem, one main issue is how to generate consensus node's embeddings for both source and target graphs that best serve graph matching tasks. In addition, it is also challenging to incorporate the discrete one-to-one matching constraints into the differentiable correspondence solver in deep matching network. To address these issues, we propose a novel Graph Adversarial Matching Network (GAMnet) for graph matching problem. GAMnet integrates graph adversarial embedding and graph matching simultaneously in a unified end-to-end network which aims to adaptively learn distribution consistent and domain invariant embeddings for GM tasks. Also, GAMnet exploits sparse GM optimization as correspondence solver which is differentiable and can also incorporate discrete one-to-one matching constraints approximately in natural in the final matching prediction. Experimental results on three public benchmarks demonstrate the effectiveness and benefits of the proposed GAMnet.

[1]  Minyi Guo,et al.  GraphGAN: Graph Representation Learning with Generative Adversarial Nets , 2017, AAAI.

[2]  Chris H. Q. Ding,et al.  Orthogonal nonnegative matrix t-factorizations for clustering , 2006, KDD '06.

[3]  Junjun Jiang,et al.  Image Matching from Handcrafted to Deep Features: A Survey , 2020, International Journal of Computer Vision.

[4]  Fredrik Kahl,et al.  Optimal correspondences from pairwise constraints , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  Gui-Song Xia,et al.  Deep Graph Matching under Quadratic Constraint , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Joan Bruna,et al.  REVISED NOTE ON LEARNING QUADRATIC ASSIGNMENT WITH GRAPH NEURAL NETWORKS , 2018, 2018 IEEE Data Science Workshop (DSW).

[7]  Fernando De la Torre,et al.  Factorized Graph Matching , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Chris H. Q. Ding,et al.  Nonnegative Matrix Factorization for Combinatorial Optimization: Spectral Clustering, Graph Matching, and Clique Finding , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[9]  Junchi Yan,et al.  Layered Neighborhood Expansion for Incremental Multiple Graph Matching , 2020, European Conference on Computer Vision.

[10]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[11]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[12]  Martial Hebert,et al.  Unsupervised Learning for Graph Matching , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Xiangliang Zhang,et al.  Improving Cross-lingual Entity Alignment via Optimal Transport , 2019, IJCAI.

[14]  Richard Sinkhorn A Relationship Between Arbitrary Positive Matrices and Doubly Stochastic Matrices , 1964 .

[15]  Baoxin Li,et al.  Determinant Regularization for Gradient-Efficient Graph Matching , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Terry Caelli,et al.  An eigenspace projection clustering method for inexact graph matching , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Alexander J. Smola,et al.  Learning Graph Matching , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Xin Li,et al.  Domain-Adversarial Network Alignment , 2019, IEEE Transactions on Knowledge and Data Engineering.

[19]  Junchi Yan,et al.  Combinatorial Learning of Robust Deep Graph Matching: An Embedding Based Approach , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Heinrich Müller,et al.  SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[22]  Yidong Li,et al.  Learning Combinatorial Solver for Graph Matching , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Zhoujun Li,et al.  Adversarial Learning for Weakly-Supervised Social Network Alignment , 2019, AAAI.

[24]  Jean Ponce,et al.  SPair-71k: A Large-scale Benchmark for Semantic Correspondence , 2019, ArXiv.

[25]  Chris H. Q. Ding,et al.  Nonnegative Orthogonal Graph Matching , 2017, AAAI.

[26]  Shuwen Yang,et al.  DANE: Domain Adaptive Network Embedding , 2019, IJCAI.

[27]  Junchi Yan,et al.  Learning Combinatorial Embedding Networks for Deep Graph Matching , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[28]  Cristian Sminchisescu,et al.  Deep Learning of Graph Matching , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[30]  Jean Ponce,et al.  Learning Graphs to Match , 2013, 2013 IEEE International Conference on Computer Vision.

[31]  Bo Jiang,et al.  Graph matching based on spectral embedding with missing value , 2012, Pattern Recognit..

[32]  Minsu Cho,et al.  Reweighted Random Walks for Graph Matching , 2010, ECCV.

[33]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[34]  Hongyuan Zha,et al.  A Short Survey of Recent Advances in Graph Matching , 2016, ICMR.

[35]  Lina Yao,et al.  Adversarially Regularized Graph Autoencoder , 2018, IJCAI.

[36]  Bo Jiang,et al.  MGARL: Multiple Graph Adversarial Regularized Learning , 2021, 2021 IEEE International Conference on Multimedia and Expo (ICME).

[37]  Shikui Tu,et al.  IA-GM: A Deep Bidirectional Learning Method for Graph Matching , 2021, AAAI.

[38]  Georg Martius,et al.  Deep Graph Matching via Blackbox Differentiation of Combinatorial Solvers , 2020, ECCV.

[39]  Jin Tang,et al.  GLMNet: Graph Learning-Matching Networks for Feature Matching , 2019, ArXiv.

[40]  Jitendra Malik,et al.  Poselets: Body part detectors trained using 3D human pose annotations , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[41]  Baoxin Li,et al.  Learning deep graph matching with channel-independent embedding and Hungarian attention , 2020, ICLR.

[42]  Dan Wang,et al.  Adversarial Network Embedding , 2017, AAAI.

[43]  Nils M. Kriege,et al.  Deep Graph Matching Consensus , 2020, ICLR.

[44]  Junchi Yan,et al.  Neural Graph Matching Network: Learning Lawler’s Quadratic Assignment Problem With Extension to Hypergraph and Multiple-Graph Matching , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.