Testing correlation of unlabeled random graphs

We study the problem of detecting the edge correlation between two random graphs with $n$ unlabeled nodes. This is formalized as a hypothesis testing problem, where under the null hypothesis, the two graphs are independently generated; under the alternative, the two graphs are edge-correlated under some latent node correspondence, but have the same marginal distributions as the null. For both Gaussian-weighted complete graphs and dense Erdős-Renyi graphs (with edge probability $n^{-o(1)}$), we determine the sharp threshold at which the optimal testing error probability exhibits a phase transition from zero to one as $n\to \infty$. For sparse Erdős-Renyi graphs with edge probability $n^{-\Omega(1)}$, we determine the threshold within a constant factor. The proof of the impossibility results is an application of the conditional second-moment method, where we bound the truncated second moment of the likelihood ratio by carefully conditioning on the typical behavior of the intersection graph (consisting of edges in both observed graphs) and taking into account the cycle structure of the induced random permutation on the edges. Notably, in the sparse regime, this is accomplished by leveraging the pseudoforest structure of subcritical Erdős-Renyi graphs and a careful enumeration of subpseudoforests that can be assembled from short orbits of the edge permutation.

[1]  W. F. Kibble An extension of a theorem of Mehler's on Hermite polynomials , 1945, Mathematical Proceedings of the Cambridge Philosophical Society.

[2]  R. Otter The Number of Trees , 1948 .

[3]  B. Harshbarger An Introduction to Probability Theory and its Applications, Volume I , 1958 .

[4]  F. T. Wright,et al.  A Bound on Tail Probabilities for Quadratic Forms in Independent Random Variables , 1971 .

[5]  G. Egorychev Integral representation and the computation of combinatorial sums , 1984 .

[6]  R. Arratia,et al.  The Cycle Structure of Random Permutations , 1992 .

[7]  Panos M. Pardalos,et al.  The Quadratic Assignment Problem: A Survey and Recent Developments , 1993, Quadratic Assignment and Related Problems.

[8]  V. V. Petrov Limit Theorems of Probability Theory: Sequences of Independent Random Variables , 1995 .

[9]  Mario Vento,et al.  Thirty Years Of Graph Matching In Pattern Recognition , 2004, Int. J. Pattern Recognit. Artif. Intell..

[10]  Jitendra Malik,et al.  Shape matching and object recognition using low distortion correspondences , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Andrew Y. Ng,et al.  Robust Textual Inference via Graph Matching , 2005, HLT.

[12]  Eli Upfal,et al.  Probability and Computing: Randomized Algorithms and Probabilistic Analysis , 2005 .

[13]  Jianbo Shi,et al.  Balanced Graph Matching , 2006, NIPS.

[14]  Alan M. Frieze,et al.  Random graphs , 2006, SODA '06.

[15]  Vitaly Shmatikov,et al.  Robust De-anonymization of Large Sparse Datasets , 2008, 2008 IEEE Symposium on Security and Privacy (sp 2008).

[16]  Bonnie Berger,et al.  Global alignment of multiple protein interaction networks with application to functional orthology detection , 2008, Proceedings of the National Academy of Sciences.

[17]  A. Hoorfar,et al.  INEQUALITIES ON THE LAMBERTW FUNCTION AND HYPERPOWER FUNCTION , 2008 .

[18]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[19]  Philippe Flajolet,et al.  Analytic Combinatorics , 2009 .

[20]  Vitaly Shmatikov,et al.  De-anonymizing Social Networks , 2009, 2009 30th IEEE Symposium on Security and Privacy.

[21]  Eric T. Harley,et al.  Fast Approximate Quadratic Programming for Large (Brain) Graph Matching , 2011 .

[22]  Matthias Grossglauser,et al.  On the privacy of anonymized networks , 2011, KDD.

[23]  John M. Conroy,et al.  Large (Brain) Graph Matching via Fast Approximate Quadratic Programming , 2011 .

[24]  Lorenzo Livi,et al.  The graph matching problem , 2012, Pattern Analysis and Applications.

[25]  Ying Wang,et al.  Message-Passing Algorithms for Sparse Network Alignment , 2009, TKDD.

[26]  E. Arias-Castro,et al.  Community Detection in Sparse Random Networks , 2013, 1308.2955.

[27]  Yu. I. Ingster,et al.  Detection of a sparse submatrix of a high-dimensional noisy matrix , 2011, 1109.0898.

[28]  M. Rudelson,et al.  Hanson-Wright inequality and sub-gaussian concentration , 2013 .

[29]  E. Arias-Castro,et al.  Community detection in dense random networks , 2014 .

[30]  Maxim Sviridenko,et al.  Maximum Quadratic Assignment Problem: Reduction from Maximum Label Cover and LP-based Approximation Algorithm , 2010, TALG.

[31]  Elchanan Mossel,et al.  Reconstruction and estimation in the planted partition model , 2012, Probability Theory and Related Fields.

[32]  Avraham Adler,et al.  Lambert-W Function , 2015 .

[33]  Daniel Cullina,et al.  Improved Achievability and Converse Bounds for Erdos-Renyi Graph Matching , 2016, SIGMETRICS.

[34]  Jess Banks,et al.  Information-theoretic thresholds for community detection in sparse networks , 2016, COLT.

[35]  A. Frieze,et al.  Introduction to Random Graphs , 2016 .

[36]  Daniel Cullina,et al.  Exact alignment recovery for correlated Erdos Renyi graphs , 2017, ArXiv.

[37]  H. Vincent Poor,et al.  Partial Recovery of Erdðs-Rényi Graph Alignment via k-Core Alignment , 2019, Proc. ACM Meas. Anal. Comput. Syst..

[38]  Galen Reeves,et al.  The All-or-Nothing Phenomenon in Sparse Linear Regression , 2019, COLT.

[39]  Jiaming Xu,et al.  Spectral Graph Matching and Regularized Quadratic Relaxations II: Erdős-Rényi Graphs and Universality , 2019, Foundations of Computational Mathematics.

[40]  Tselil Schramm,et al.  (Nearly) Efficient Algorithms for the Graph Matching Problem on Correlated Random Graphs , 2018, NeurIPS.

[41]  Elchanan Mossel,et al.  Seeded graph matching via large neighborhood statistics , 2018, SODA.

[42]  Jiaming Xu,et al.  Spectral Graph Matching and Regularized Quadratic Relaxations I: The Gaussian Model , 2019, ArXiv.

[43]  Daniel Cullina,et al.  Analysis of a Canonical Labeling Algorithm for the Alignment of Correlated Erdős-Rényi Graphs , 2018, Abstracts of the 2019 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer Systems.

[44]  Béla Bollobás,et al.  Dense subgraphs in random graphs , 2018, Discret. Appl. Math..

[45]  Laurent Massoulié,et al.  Spectral alignment of correlated Gaussian random matrices , 2019, ArXiv.

[46]  Daniel Cullina,et al.  Database Alignment with Gaussian Features , 2019, AISTATS.

[47]  Laurent Massouli'e,et al.  Partial Recovery in the Graph Alignment Problem , 2020, Oper. Res..

[48]  Jiaming Xu,et al.  Efficient random graph matching via degree profiles , 2018, Probability Theory and Related Fields.

[49]  Luca Ganassali,et al.  From tree matching to sparse graph alignment , 2020, COLT.

[50]  Prateek Mittal,et al.  Partial Recovery of Erdős-Rényi Graph Alignment via k-Core Alignment , 2020, Abstracts of the 2020 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer Systems.

[51]  Miklós Z. Rácz,et al.  Correlated randomly growing graphs , 2020, The Annals of Applied Probability.

[52]  Sophie H. Yu,et al.  Settling the Sharp Reconstruction Thresholds of Random Graph Matching , 2021, 2021 IEEE International Symposium on Information Theory (ISIT).

[53]  Spectral alignment of correlated Gaussian matrices , 2019, Advances in Applied Probability.