Wasserstein-Based Graph Alignment

We propose a novel method for comparing non-aligned graphs of different sizes, based on the Wasserstein distance between graph signal distributions induced by the respective graph Laplacian matrices. Specifically, we cast a new formulation for the one-to-many graph alignment problem, which aims at matching a node in the smaller graph with one or more nodes in the larger graph. By integrating optimal transport in our graph comparison framework, we generate both a structurally-meaningful graph distance, and a signal transportation plan that models the structure of graph data. The resulting alignment problem is solved with stochastic gradient descent, where we use a novel Dykstra operator to ensure that the solution is a one-to-many (soft) assignment matrix. We demonstrate the performance of our novel framework on graph alignment and graph classification, and we show that our method leads to significant improvements with respect to the state-of-the-art algorithms for each of these tasks.

[1]  Yaron Lipman,et al.  DS++ , 2017, ACM Trans. Graph..

[2]  Martial Hebert,et al.  An Integer Projected Fixed Point Method for Graph Matching and MAP Inference , 2009, NIPS.

[3]  Richard Sinkhorn.  A Relationship Between Arbitrary Positive Matrices and Doubly Stochastic Matrices , 1964 .

[4]  Steven Gold,et al.  A Graduated Assignment Algorithm for Graph Matching , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Nils M. Kriege,et al.  On Valid Optimal Assignment Kernels and Applications to Graph Classification , 2016, NIPS.

[6]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[7]  Jianbo Shi,et al.  Balanced Graph Matching , 2006, NIPS.

[8]  Facundo Mémoli,et al.  Gromov–Wasserstein Distances and the Metric Approach to Object Matching , 2011, Found. Comput. Math..

[9]  Subhransu Maji,et al.  Improved Bilinear Pooling with CNNs , 2017, BMVC.

[10]  R. Dykstra.  An Algorithm for Restricted Least Squares Regression , 1983 .

[11]  Hongyuan Zha,et al.  A Short Survey of Recent Advances in Graph Matching , 2016, ICMR.

[12]  Asuka Takatsu.  Wasserstein geometry of Gaussian measures , 2011 .

[13]  Nicolas Courty,et al.  Optimal Transport for structured data , 2018, ArXiv.

[14]  Lawrence Carin,et al.  Scalable Gromov-Wasserstein Learning for Graph Partitioning and Matching , 2019, NeurIPS.

[15]  Gabriel Peyré,et al.  Iterative Bregman Projections for Regularized Transportation Problems , 2014, SIAM J. Sci. Comput..

[16]  Didrik Nielsen,et al.  Variational Adaptive-Newton Method for Explorative Learning , 2017, ArXiv.

[17]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[18]  Leonhard Held,et al.  Gaussian Markov Random Fields: Theory and Applications , 2005 .

[19]  Terry Caelli,et al.  An eigenspace projection clustering method for inexact graph matching , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Pascal Frossard,et al.  Learning Graphs From Data: A Signal Representation Perspective , 2018, IEEE Signal Processing Magazine.

[21]  Emmanuel Müller,et al.  NetLSD: Hearing the Shape of a Graph , 2018, KDD.

[22]  A. Rakotomamonjy,et al.  Optimal transport with Laplacian regularization , 2014, NIPS 2014.

[23]  Sanjay Ranka,et al.  Learning Permutations with Sinkhorn Policy Gradient , 2018, ArXiv.

[24]  Sanjiv Kumar,et al.  On the Convergence of Adam and Beyond , 2018 .

[25]  Fernando De la Torre,et al.  Factorized Graph Matching , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Francis R. Bach,et al.  Many-to-Many Graph Matching: a Continuous Relaxation Approach , 2010, ECML/PKDD.

[27]  Heinz H. Bauschke,et al.  Dykstras algorithm with bregman projections: A convergence proof , 2000 .

[28]  Gabriel Peyré,et al.  Learning Generative Models with Sinkhorn Divergences , 2017, AISTATS.

[29]  Pinar Yanardag,et al.  Deep Graph Kernels , 2015, KDD.

[30]  Alessandro Rudi,et al.  Differential Properties of Sinkhorn Approximation for Learning with Wasserstein Distance , 2018, NeurIPS.

[31]  Baoxin Li,et al.  Generalizing Graph Matching beyond Quadratic Assignment Model , 2018, NeurIPS.

[32]  Y. Aflalo,et al.  On convex relaxation of graph isomorphism , 2015, Proceedings of the National Academy of Sciences.

[33]  Kaspar Riesen,et al.  Fast Suboptimal Algorithms for the Computation of Graph Edit Distance , 2006, SSPR/SPR.

[34]  Guillermo Sapiro,et al.  On spectral properties for graph matching and graph isomorphism problems , 2014, ArXiv.

[35]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[36]  Yihong Gong,et al.  Graph Matching via Multiplicative Update Algorithm , 2017, NIPS.

[37]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[38]  Minsu Cho,et al.  Reweighted Random Walks for Graph Matching , 2010, ECCV.

[39]  Michalis Vazirgiannis,et al.  Matching Node Embeddings for Graph Similarity , 2017, AAAI.

[40]  Jiao Gu,et al.  Spectral distances on graphs , 2015, Discret. Appl. Math..

[41]  Christoph Schnörr,et al.  Probabilistic Subgraph Matching Based on Convex Relaxation , 2005, EMMCVPR.

[42]  Xavier Bresson,et al.  Robust Principal Component Analysis on Graphs , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[43]  Pascal Frossard,et al.  Learning Laplacian Matrix in Smooth Graph Signal Representations , 2014, IEEE Transactions on Signal Processing.

[44]  Scott W. Linderman,et al.  Learning Latent Permutations with Gumbel-Sinkhorn Networks , 2018, ICLR.