DETECTION THRESHOLDS FOR THE β -MODEL ON SPARSE GRAPHS

In this paper, we study sharp thresholds for detecting sparse signals in β-models for potentially sparse random graphs. The results demonstrate interesting interplay between graph sparsity, signal sparsity and signal strength. In regimes of moderately dense signals, irrespective of graph sparsity, the detection thresholds mirror corresponding results in independent Gaussian sequence problems. For sparser signals, extreme graph sparsity implies that all tests are asymptotically powerless, irrespective of the signal strength. On the other hand, sharp detection thresholds are obtained, up to matching constants, on denser graphs. The phase transitions mentioned above are sharp. As a crucial ingredient, we study a version of the higher criticism test which is provably sharp up to optimal constants in the regime of sparse signals. The theoretical results are further verified by numerical simulations.

[1]  L. Addario-Berry,et al.  On Combinatorial Testing Problems 1 , 2010 .

[2]  Xiaoming Huo,et al.  Near-optimal detection of geometric objects by fast multiscale methods , 2005, IEEE Transactions on Information Theory.

[3]  D. Donoho,et al.  Higher criticism for detecting sparse heterogeneous mixtures , 2004, math/0410072.

[4]  S. Lauritzen.  Exchangeable Rasch Matrices∗ , 2007 .

[5]  Patrick J. Wolfe,et al.  Null models for network data , 2012, ArXiv.

[6]  Xihong Lin,et al.  HYPOTHESIS TESTING FOR HIGH-DIMENSIONAL SPARSE BINARY REGRESSION. , 2013, Annals of statistics.

[7]  E. Candès,et al.  Global testing under sparse alternatives: ANOVA, multiple comparisons and the higher criticism , 2010, 1007.1434.

[8]  P. Bickel,et al.  The method of moments and degree distributions for network models , 2011, 1202.5101.

[9]  Yu. I. Ingster,et al.  Nonparametric Goodness-of-Fit Testing Under Gaussian Models , 2002 .

[10]  S. Fienberg,et al.  Categorical Data Analysis of Single Sociometric Relations , 1981 .

[11]  Xihong Lin,et al.  The Generalized Higher Criticism for Testing SNP-Set Effects in Genetic Association Studies , 2017, Journal of the American Statistical Association.

[12]  Allan Sly,et al.  Random graphs with a given degree sequence , 2010, 1005.1136.

[13]  P. Hall,et al.  Innovated Higher Criticism for Detecting Sparse Signals in Correlated Noise , 2009, 0902.3837.

[14]  Yu. I. Ingster,et al.  Detection boundary in sparse regression , 2010, 1009.1706.

[15]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[16]  Vishesh Karwa,et al.  Inference using noisy degrees: Differentially private $\beta$-model and synthetic graphs , 2012, 1205.4697.

[17]  T. Yan,et al.  Asymptotics in Undirected Random Graph Models Parameterized by the Strengths of Vertices , 2015 .

[18]  E. Arias-Castro,et al.  Community Detection in Random Networks , 2013, 1302.7099.

[19]  Persi Diaconis,et al.  A Sequential Importance Sampling Algorithm for Generating Random Graphs with Prescribed Degrees , 2011, Internet Math..

[20]  Cristopher Moore,et al.  Model selection for degree-corrected block models , 2012, Journal of statistical mechanics.

[21]  E. Arias-Castro,et al.  The Sparse Poisson Means Model , 2015, 1505.01247.

[22]  Alexander I. Barvinok,et al.  The number of graphs and a random graph with a given degree sequence , 2010, Random Struct. Algorithms.

[23]  Akimichi Takemura,et al.  MATHEMATICAL ENGINEERING TECHNICAL REPORTS Connecting Tables with Zero-One Entries by a Subset of a Markov Basis , 2009 .

[24]  Ming Yuan,et al.  Rate-Optimal Detection of Very Short Signal Segments , 2014, ArXiv.

[25]  Stephen E. Fienberg,et al.  Algebraic Statistics for a Directed Random Graph Model with Reciprocation , 2009, 0909.0073.

[26]  Hisayuki Hara,et al.  Graver basis for an undirected graph and its application to testing the beta model of random graphs , 2011, 1102.2583.

[27]  C. Hillar,et al.  Maximum entropy distributions on graphs , 2013, 1301.3321.

[28]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[29]  Hong Qin,et al.  Asymptotic normality in the maximum entropy models on graphs with an increasing number of parameters , 2013, J. Multivar. Anal..

[30]  P. Holland,et al.  An Exponential Family of Probability Distributions for Directed Graphs , 1981 .

[31]  E. Candès,et al.  Searching for a trail of evidence in a maze , 2007, math/0701668.

[32]  Steven M. Goodreau,et al.  Advances in exponential random graph (p*) models applied to a large social network , 2007, Soc. Networks.

[33]  Garry Robins,et al.  An introduction to exponential random graph (p*) models for social networks , 2007, Soc. Networks.

[34]  T. Yan,et al.  A central limit theorem in the β-model for undirected random graphs with a diverging number of vertices , 2012, 1202.3307.

[35]  Mark E. J. Newman,et al.  Stochastic blockmodels and community structure in networks , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[36]  E. Arias-Castro,et al.  Community Detection in Sparse Random Networks , 2013, 1308.2955.