A Convergent Single-Loop Algorithm for Relaxation of Gromov-Wasserstein in Graph Data

In this work, we present the Bregman Alternating Projected Gradient (BAPG) method, a single-loop algorithm that offers an approximate solution to the Gromov-Wasserstein (GW) distance. We introduce a novel relaxation technique that balances accuracy and computational efficiency, albeit with some compromises in the feasibility of the coupling map. Our analysis is based on the observation that the GW problem satisfies the Luo-Tseng error bound condition, which relates to estimating the distance of a point to the critical point set of the GW problem based on the optimality residual. This observation allows us to provide an approximation bound for the distance between the fixed-point set of BAPG and the critical point set of GW. Moreover, under a mild technical assumption, we can show that BAPG converges to its fixed point set. The effectiveness of BAPG has been validated through comprehensive numerical experiments in graph alignment and partition tasks, where it outperforms existing methods in terms of both solution quality and wall-clock time.

[1]  F. Tsung,et al.  Robust Attributed Graph Alignment via Joint Structure Learning and Optimal Transport , 2023, ArXiv.

[2]  L. Carin,et al.  Representing Graphs via Gromov-Wasserstein Factorization , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Jiliang Tang,et al.  Graph Neural Networks for Multimodal Single-Cell Data Integration , 2022, KDD.

[4]  Mikael Johansson,et al.  A Fast and Accurate Splitting Method for Optimal Transport: Analysis and Implementation , 2021, ICLR.

[5]  N. Courty,et al.  Semi-relaxed Gromov Wasserstein divergence with applications on graphs , 2021, ArXiv.

[6]  Jiawei Zhang,et al.  A Global Dual Error Bound and Its Application to the Analysis of Linearly Constrained Nonconvex Optimization , 2020, SIAM J. Optim..

[7]  Hanghang Tong,et al.  Balancing Consistency and Disparity in Network Alignment , 2021, KDD.

[8]  Nicolas Courty,et al.  Online Graph Dictionary Learning , 2021, ICML.

[9]  Hong Cheng,et al.  Mask-GVAE: Blind Denoising Graphs via Partition , 2021, WWW.

[10]  Hongyuan Zha,et al.  Learning Graphons via Structured Gromov-Wasserstein Barycenters , 2020, AAAI.

[11]  François-Xavier Vialard,et al.  The Unbalanced Gromov Wasserstein Distance: Conic Formulation and Relaxation , 2020, NeurIPS.

[12]  Samir Chowdhury,et al.  Generalized Spectral Clustering via Gromov-Wasserstein Learning , 2020, AISTATS.

[13]  Jundong Li,et al.  Unsupervised Graph Alignment with Wasserstein Distance Discriminator , 2021, KDD.

[14]  Nicolas Courty,et al.  POT: Python Optimal Transport , 2021, J. Mach. Learn. Res..

[15]  Kai Cao,et al.  Manifold alignment for heterogeneous single-cell multi-omics data integration using Pamona , 2020, bioRxiv.

[16]  Honglei Zhang,et al.  Dirichlet Graph Variational Autoencoder , 2020, NeurIPS.

[17]  Yiguang Hong,et al.  Unsupervised topological alignment for single-cell multi-omics integration , 2020, bioRxiv.

[18]  Method of the Year 2019: Single-cell multimodal omics , 2020, Nature Methods.

[19]  Tara Abrishami,et al.  Geometry of Graph Partitions via Optimal Transport , 2019, SIAM J. Sci. Comput..

[20]  Marco Cuturi,et al.  Computational Optimal Transport: With Applications to Data Science , 2019 .

[21]  Lawrence Carin,et al.  Scalable Gromov-Wasserstein Learning for Graph Partitioning and Matching , 2019, NeurIPS.

[22]  Stefanie Jegelka,et al.  Learning Generative Models across Incomparable Spaces , 2019, ICML.

[23]  Hongyuan Zha,et al.  Gromov-Wasserstein Learning for Graph Matching and Node Embedding , 2019, ICML.

[24]  Samir Chowdhury,et al.  The Gromov-Wasserstein distance between networks and stable network invariants , 2018, Information and Inference: A Journal of the IMA.

[25]  Nicolas Courty,et al.  Optimal Transport for structured data with application on graphs , 2018, ICML.

[26]  Nicolas Courty,et al.  Fused Gromov-Wasserstein distance for structured objects: theoretical foundations and mathematical properties , 2018, Algorithms.

[27]  Anthony Man-Cho So,et al.  A unified approach to error bounds for structured convex optimization problems , 2015, Mathematical Programming.

[28]  Vladimir G. Kim,et al.  Entropic metric alignment for correspondence problems , 2016, ACM Trans. Graph..

[29]  Gabriel Peyré,et al.  Gromov-Wasserstein Averaging of Kernel and Distance Matrices , 2016, ICML.

[30]  Dimitri P. Bertsekas,et al.  Stochastic First-Order Methods with Random Constraint Projection , 2016, SIAM J. Optim..

[31]  Alexandre M. Bayen,et al.  Efficient Bregman projections onto the simplex , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[32]  Facundo Mémoli,et al.  The Gromov-Wasserstein Distance: A Brief Overview , 2014, Axioms.

[33]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[34]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[35]  Angelia Nedic,et al.  Random algorithms for convex minimization problems , 2011, Math. Program..

[36]  Facundo Mémoli,et al.  Gromov–Wasserstein Distances and the Metric Approach to Object Matching , 2011, Found. Comput. Math..

[37]  Minsu Cho,et al.  Reweighted Random Walks for Graph Matching , 2010, ECCV.

[38]  James Bailey,et al.  Information Theoretic Measures for Clusterings Comparison: Variants, Properties, Normalization and Correction for Chance , 2010, J. Mach. Learn. Res..

[39]  Hédy Attouch,et al.  Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Lojasiewicz Inequality , 2008, Math. Oper. Res..

[40]  Martial Hebert,et al.  An Integer Projected Fixed Point Method for Graph Matching and MAP Inference , 2009, NIPS.

[41]  Facundo Mémoli,et al.  Spectral Gromov-Wasserstein distances for shape matching , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[42]  Panos M. Pardalos,et al.  Quadratic Assignment Problem , 1997, Encyclopedia of Optimization.

[43]  Jean-Loup Guillaume,et al.  Fast unfolding of communities in large networks , 2008, 0803.0476.

[44]  Martin Rosvall,et al.  Maps of random walks on complex networks reveal community structure , 2007, Proceedings of the National Academy of Sciences.

[45]  Ben Taskar,et al.  Word Alignment via Quadratic Assignment , 2006, NAACL.

[46]  Martial Hebert,et al.  A spectral technique for correspondence problems using pairwise constraints , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[47]  M. Newman,et al.  Finding community structure in very large networks. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[48]  Guillermo Sapiro,et al.  Comparing point clouds , 2004, SGP '04.

[49]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[50]  Heinz H. Bauschke,et al.  Projection algorithms and monotone operators , 1996 .

[51]  Paul Tseng,et al.  Error Bound and Convergence Analysis of Matrix Splitting Algorithms for the Affine Variational Inequality Problem , 1992, SIAM J. Optim..