Learning Graphons via Structured Gromov-Wasserstein Barycenters

We propose a novel and principled method to learn a nonparametric graph model called graphon, which is defined in an infinite-dimensional space and represents arbitrary-size graphs. Based on the weak regularity lemma from the theory of graphons, we leverage a step function to approximate a graphon. We show that the cut distance of graphons can be relaxed to the Gromov-Wasserstein distance of their step functions. Accordingly, given a set of graphs generated by an underlying graphon, we learn the corresponding step function as the Gromov-Wasserstein barycenter of the given graphs. Furthermore, we develop several enhancements and extensions of the basic algorithm, $e.g.$, the smoothed Gromov-Wasserstein barycenter for guaranteeing the continuity of the learned graphons and the mixed Gromov-Wasserstein barycenters for learning multiple structured graphons. The proposed approach overcomes drawbacks of prior state-of-the-art methods, and outperforms them on both synthetic and real-world data. The code is available at this https URL.

[1]  Richard Sinkhorn,et al.  Concerning nonnegative matrices and doubly stochastic matrices , 1967 .

[2]  X ZhengAlice,et al.  A Survey of Statistical Network Models , 2010 .

[3]  Francesca Parise,et al.  Graphon Games , 2018, EC.

[4]  Gabriel Peyré,et al.  Computational Optimal Transport , 2018, Found. Trends Mach. Learn..

[5]  Kristian Kersting,et al.  TUDataset: A collection of benchmark datasets for learning with graphs , 2020, ArXiv.

[6]  Edoardo M. Airoldi,et al.  A Survey of Statistical Network Models , 2009, Found. Trends Mach. Learn..

[7]  S. Janson Graphons, cut norm and distance, couplings and rearrangements , 2010, 1009.2376.

[8]  T. Snijders,et al.  Estimation and Prediction for Stochastic Blockstructures , 2001 .

[9]  Gabriel Peyré,et al.  Gromov-Wasserstein Averaging of Kernel and Distance Matrices , 2016, ICML.

[10]  Lawrence Carin,et al.  Scalable Gromov-Wasserstein Learning for Graph Partitioning and Matching , 2019, NeurIPS.

[11]  Lawrence Carin,et al.  Hierarchical Optimal Transport for Robust Multi-View Learning , 2020, ArXiv.

[12]  Jason Altschuler,et al.  Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration , 2017, NIPS.

[13]  Nicolas Courty,et al.  Sliced Gromov-Wasserstein , 2019, NeurIPS.

[14]  Asuman Ozdaglar,et al.  Graphon Games: A Statistical Framework for Network Games and Interventions , 2018, SSRN Electronic Journal.

[15]  M. Jackson,et al.  Games on Networks , 2014 .

[16]  Nicolas Courty,et al.  Optimal Transport for structured data with application on graphs , 2018, ICML.

[17]  Jiaming Xu,et al.  Rates of Convergence of Spectral Methods for Graphon Estimation , 2017, ICML.

[18]  M. Bálek,et al.  Large Networks and Graph Limits , 2022 .

[19]  P. Diaconis,et al.  Graph limits and exchangeable random graphs , 2007, 0712.2749.

[20]  J. Daudin,et al.  Classification and estimation in the Stochastic Block Model based on the empirical degrees , 2011, 1110.6517.

[21]  C. Villani Optimal Transport: Old and New , 2008 .

[22]  Hongteng Xu,et al.  Gromov-Wasserstein Factorization Models for Graph Clustering , 2019, AAAI.

[23]  Antoni Calvó-Armengol,et al.  Centre De Referència En Economia Analítica Barcelona Economics Working Paper Series Working Paper Nº 178 Who's Who in Networks. Wanted: the Key Player Who's Who in Networks. Wanted: the Key Player Barcelona Economics Wp Nº 178 , 2022 .

[24]  Facundo Mémoli,et al.  Gromov–Wasserstein Distances and the Metric Approach to Object Matching , 2011, Found. Comput. Math..

[25]  Alan M. Frieze,et al.  Quick Approximation to Matrices and Applications , 1999, Comb..

[26]  Hongyuan Zha,et al.  A Fast Proximal Point Method for Computing Exact Wasserstein Distance , 2018, UAI.

[27]  V. Sós,et al.  Convergent Sequences of Dense Graphs I: Subgraph Frequencies, Metric Properties and Testing , 2007, math/0702004.

[28]  Nicolas Courty,et al.  Fused Gromov-Wasserstein distance for structured objects: theoretical foundations and mathematical properties , 2018, Algorithms.

[29]  A. Nagurney Network Economics: A Variational Inequality Approach , 1992 .

[30]  Peter D. Hoff,et al.  Latent Space Approaches to Social Network Analysis , 2002 .

[31]  Hongyuan Zha,et al.  Gromov-Wasserstein Learning for Graph Matching and Node Embedding , 2019, ICML.

[32]  S. Janson,et al.  Graph limits and exchangeable random graphs , 2007, 0712.2749.

[33]  Eric D. Kolaczyk,et al.  Statistical Analysis of Network Data: Methods and Models , 2009 .

[34]  Pinar Yanardag,et al.  Deep Graph Kernels , 2015, KDD.

[35]  Edoardo M. Airoldi,et al.  A Consistent Histogram Estimator for Exchangeable Graph Models , 2014, ICML.

[36]  Marianna Pensky,et al.  Dynamic network models and graphon estimation , 2016, The Annals of Statistics.

[37]  Samir Chowdhury,et al.  The Gromov-Wasserstein distance between networks and stable network invariants , 2018, Information and Inference: A Journal of the IMA.

[38]  S. Chatterjee,et al.  Matrix estimation by Universal Singular Value Thresholding , 2012, 1212.1247.

[39]  Edoardo M. Airoldi,et al.  Stochastic blockmodel approximation of a graphon: Theory and consistent estimation , 2013, NIPS.

[40]  Alejandro Ribeiro,et al.  The Graphon Fourier Transform , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[41]  Edoardo M. Airoldi,et al.  Graphlet decomposition of a weighted network , 2012, AISTATS.

[42]  Shuang Gao,et al.  Graphon Control of Large-Scale Networks of Linear Systems , 2018, IEEE Transactions on Automatic Control.

[43]  Fan Chung Graham,et al.  On the Spectra of General Random Graphs , 2011, Electron. J. Comb..

[44]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, 2009 IEEE International Symposium on Information Theory.

[45]  Santiago Segarra,et al.  Centrality Measures for Graphons: Accounting for Uncertainty in Networks , 2017, IEEE Transactions on Network Science and Engineering.

[46]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.