Phase Transitions for Detecting Latent Geometry in Random Graphs

Random graphs with latent geometric structure are popular models of social and biological networks, with applications ranging from network user profiling to circuit design. These graphs are also of purely theoretical interest within computer science, probability and statistics. A fundamental initial question regarding these models is: when are these random graphs affected by their latent geometry and when are they indistinguishable from simpler models without latent structure, such as the Erdős-Renyi graph $\mathcal{G}(n, p)$? We address this question for two of the most well-studied models of random graphs with latent geometry -- the random intersection and random geometric graph. Our results are as follows: (1) we prove that the random intersection graph converges in total variation to $\mathcal{G}(n, p)$ when $d = \tilde{\omega}(n^3)$, and does not if $d = o(n^3)$, resolving an open problem in Fill et al. (2000), Rybarczyk (2011) and Kim et al. (2018); (2) we provide conditions under which the matrix of intersection sizes of random family of sets converges in total variation to a symmetric matrix with independent Poisson entries, yielding the first total variation convergence result for $\tau$-random intersection graphs to $\mathcal{G}(n, p)$; and (3) we show that the random geometric graph on $\mathbb{S}^{d - 1}$ with edge density $p$ converges in total variation to $\mathcal{G}(n, p)$ when $d = \tilde{\omega}\left(\min\{ pn^3, p^2 n^{7/2} \} \right)$, yielding the first progress towards a conjecture of Bubeck et al. (2016). The first of these three results was obtained simultaneously and independently by Bubeck, Racz and Richey.

[1]  F. Ball,et al.  Epidemics on random intersection graphs , 2010, 1011.4242.

[2]  Katarzyna Rybarczyk,et al.  Diameter, connectivity, and phase transition of the uniform random intersection graph , 2011, Discret. Math..

[3]  Stefanie Gerke,et al.  Connectivity of the uniform random intersection graph , 2008, Discret. Math..

[4]  Peter D. Hoff,et al.  Latent Space Approaches to Social Network Analysis , 2002 .

[5]  Erhard Godehardt,et al.  Two Models of Random Intersection Graphs for Classification , 2003 .

[6]  D. Marchette Random Graphs for Statistical Pattern Recognition , 2004 .

[7]  Philippe Rigollet,et al.  Complexity Theoretic Lower Bounds for Sparse Principal Component Detection , 2013, COLT.

[8]  Jerzy Jaworski,et al.  The degree of a typical vertex in generalized random intersection graph models , 2006, Discret. Math..

[9]  Paolo Santi Topology control in wireless ad hoc and sensor networks , 2005 .

[10]  G. Lugosi,et al.  Detecting Positive Correlations in a Multivariate Sample , 2012, 1202.5536.

[11]  Katarzyna Rybarczyk,et al.  Sharp Threshold Functions for Random Intersection Graphs via a Coupling Method , 2011, Electron. J. Comb..

[12]  Edward R. Scheinerman,et al.  On Random Intersection Graphs: The Subgraph Problem , 1999, Combinatorics, Probability and Computing.

[13]  Guy Bresler,et al.  Average-Case Lower Bounds for Learning Sparse Mixtures, Robust Estimation and Semirandom Adversaries , 2019, ArXiv.

[14]  Guy Bresler,et al.  Reducibility and Statistical-Computational Gaps from Secret Leakage , 2020, COLT 2020.

[15]  Wasim Huleihel,et al.  Universality of Computational Lower Bounds for Submatrix Detection , 2019, COLT.

[16]  Jeong Han Kim,et al.  On the total variation distance between the binomial random graph and the random intersection graph , 2015, Random Struct. Algorithms.

[17]  Guy Bresler,et al.  Optimal Average-Case Reductions to Sparse PCA: From Weak Assumptions to Strong Hardness , 2019, COLT.

[18]  Gábor Lugosi,et al.  Concentration Inequalities , 2008, COLT.

[19]  Ivan Nourdin,et al.  Asymptotic Behavior of Large Gaussian Correlated Wishart Matrices , 2018, Journal of Theoretical Probability.

[20]  Jens Grygierek,et al.  Poisson fluctuations for edge counts in high-dimensional random geometric graphs , 2019, 1905.11221.

[21]  Elchanan Mossel,et al.  Reconstruction and estimation in the planted partition model , 2012, Probability Theory and Related Fields.

[22]  Miklós Z. Rácz,et al.  A Smooth Transition from Wishart to GOE , 2016, 1611.05838.

[23]  Martin T. Wells,et al.  The middle-scale asymptotics of Wishart matrices , 2017, The Annals of Statistics.

[24]  Wasim Huleihel,et al.  Reducibility and Computational Lower Bounds for Problems with Planted Sparse Structure , 2018, COLT.

[25]  Andreas N. Lagerås,et al.  Epidemics on Random Graphs with Tunable Clustering , 2007, Journal of Applied Probability.

[26]  Svante Janson,et al.  Asymptotic equivalence and contiguity of some random graphs , 2008, Random Struct. Algorithms.

[27]  Alison L Gibbs,et al.  On Choosing and Bounding Probability Metrics , 2002, math/0209021.

[28]  Emmanuel Abbe,et al.  Community Detection in General Stochastic Block models: Fundamental Limits and Efficient Algorithms for Recovery , 2015, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[29]  Erhard Godehardt,et al.  Recent Progress in Complex Network Analysis: Models of Random Intersection Graphs , 2013, ECDA.

[30]  Jeffrey G. Andrews,et al.  Stochastic geometry and random graphs for the analysis and design of wireless networks , 2009, IEEE Journal on Selected Areas in Communications.

[31]  Danning Li,et al.  Approximation of Rectangular Beta-Laguerre Ensembles and Large Deviations , 2013, 1309.3882.

[32]  A. Karr Random Graphs for Statistical Pattern Recognition , 2006 .

[33]  Will Perkins Birthday Inequalities, Repulsion, and Hard Spheres , 2015, 1506.02700.

[34]  James Allen Fill,et al.  Random intersection graphs when m= w (n): an equivalence theorem relating the evolution of the G ( n, m, p ) and G ( n,P /italic>) models , 2000 .

[35]  Dudley Stark The vertex degree distribution of random intersection graphs , 2004 .

[36]  Aryeh Kontorovich,et al.  Concentration of measure without independence: a unified approach via the martingale method , 2016, ArXiv.

[37]  P. Rigollet,et al.  Optimal detection of sparse principal components in high dimension , 2012, 1202.5070.

[38]  Paul G. Spirakis,et al.  Large independent sets in general random intersection graphs , 2008, Theor. Comput. Sci..

[39]  Christoph Thaele,et al.  Gaussian fluctuations for edge counts in high-dimensional random geometric graphs , 2016, Statistics & Probability Letters.

[40]  Yihong Wu,et al.  Computational Barriers in Minimax Submatrix Detection , 2013, ArXiv.

[41]  Stephen P. Boyd,et al.  Randomized gossip algorithms , 2006, IEEE Transactions on Information Theory.

[42]  S. Sodin Tail-Sensitive Gaussian Asymptotics for Marginals of Concentrated Measures in High Dimension , 2005, math/0501382.

[43]  A. Frieze,et al.  Introduction to Random Graphs , 2016 .

[44]  Armand M. Makowski,et al.  On the random graph induced by a random key predistribution scheme under full visibility , 2008, 2008 IEEE International Symposium on Information Theory.

[45]  Valentas Kurauskas,et al.  Large Cliques in Sparse Random Intersection Graphs , 2013, Electron. J. Comb..

[46]  Sébastien Bubeck,et al.  Entropic CLT and phase transition in high-dimensional Wishart matrices , 2015, ArXiv.

[47]  Jiaming Xu,et al.  Statistical Problems with Planted Structures: Information-Theoretical and Computational Limits , 2018, Information-Theoretic Methods in Data Science.

[48]  Katarzyna Rybarczyk,et al.  Equivalence of a random intersection graph and G(n,p) , 2009, Random Struct. Algorithms.

[49]  Sébastien Bubeck,et al.  Testing for high‐dimensional geometry in random graphs , 2014, Random Struct. Algorithms.

[50]  Lars Holst,et al.  Some applications of the Stein-Chen method for proving Poisson convergence , 1989, Advances in Applied Probability.

[51]  Guy Bresler,et al.  Optimal Single Sample Tests for Structured versus Unstructured Network Data , 2018, COLT.

[52]  Mathew D. Penrose,et al.  Random Geometric Graphs , 2003 .

[53]  Erhard Godehardt,et al.  Recent Progress in Complex Network Analysis: Properties of Random Intersection Graphs , 2013, ECDA.

[54]  Bruce E. Hajek,et al.  Computational Lower Bounds for Community Detection on Random Graphs , 2014, COLT.

[55]  Sébastien Bubeck,et al.  Basic models and questions in statistical network analysis , 2016, ArXiv.

[56]  Ronen Eldan,et al.  Information and dimensionality of anisotropic random geometric graphs , 2016, ArXiv.

[57]  Mindaugas Bloznelis,et al.  Component evolution in a secure wireless sensor network , 2009, Networks.

[58]  Emilio Frazzoli,et al.  Sampling-based algorithms for optimal motion planning , 2011, Int. J. Robotics Res..

[59]  G. Lugosi,et al.  High-dimensional random geometric graphs and their clique number , 2011 .

[60]  James Allen Fill,et al.  Random intersection graphs when m=omega(n): An equivalence theorem relating the evolution of the G(n, m, p) and G(n, p) models , 2000, Random Struct. Algorithms.

[61]  Mindaugas Bloznelis,et al.  Degree and clustering coefficient in sparse random intersection graphs , 2013, 1303.3388.