Completeness of Community Structure in Networks

By defining a new measure to community structure, exclusive modularity, and based on cavity method of statistical physics, we develop a mathematically principled method to determine the completeness of community structure, which represents whether a partition that is either annotated by experts or given by a community-detection algorithm, carries complete information about community structure in the network. Our results demonstrate that the expert partition is surprisingly incomplete in some networks such as the famous political blogs network, indicating that the relation between meta-data and community structure in real-world networks needs to be re-examined. As a byproduct we find that the exclusive modularity, which introduces a null model based on the degree-corrected stochastic block model, is of independent interest. We discuss its applications as principled ways of detecting hidden structures, finding hierarchical structures without removing edges, and obtaining low-dimensional embedding of networks.

[1]  Tiago P. Peixoto.  Nonparametric Bayesian inference of the microcanonical stochastic block model. , 2016, Physical review. E.

[2]  Lada A. Adamic,et al.  The political blogosphere and the 2004 U.S. election: divided they blog , 2005, LinkKDD '05.

[3]  Kathryn B. Laskey,et al.  Stochastic blockmodels: First steps , 1983 .

[4]  F. Radicchi,et al.  Benchmark graphs for testing community detection algorithms. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[5]  Raj Rao Nadakuditi,et al.  Graph spectra and the detectability of community structure in networks , 2012, Physical review letters.

[6]  C. Lee Giles,et al.  Self-Organization and Identification of Web Communities , 2002, Computer.

[7]  M E J Newman,et al.  Fast algorithm for detecting community structure in networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[8]  M. Newman,et al.  Finding community structure in networks using the eigenvectors of matrices. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[9]  Mark E. J. Newman,et al.  Stochastic blockmodels and community structure in networks , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[10]  Konstantin Avrachenkov,et al.  Cooperative Game Theory Approaches for Network Partitioning , 2017, COCOON.

[11]  Cristopher Moore,et al.  Scalable detection of statistically significant communities and hierarchies, using message passing for modularity , 2014, Proceedings of the National Academy of Sciences.

[12]  R. Guimerà,et al.  Modularity from fluctuations in random graphs and complex networks. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[13]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[14]  M. Newman.  Communities, modules and large-scale structure in networks , 2011, Nature Physics.

[15]  Santo Fortunato,et al.  Community detection in networks: A user guide , 2016, ArXiv.

[16]  M E J Newman,et al.  Finding and evaluating community structure in networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[17]  Petter Holme,et al.  Subnetwork hierarchies of biochemical pathways , 2002, Bioinform..

[18]  Mark E. J. Newman,et al.  Structure and inference in annotated networks , 2015, Nature Communications.

[19]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[20]  W. Zachary,et al.  An Information Flow Model for Conflict and Fission in Small Groups , 1977, Journal of Anthropological Research.

[21]  Elchanan Mossel,et al.  Spectral redemption in clustering sparse networks , 2013, Proceedings of the National Academy of Sciences.

[22]  Tiago P. Peixoto.  Hierarchical block structures and high-resolution model selection in large networks , 2013, ArXiv.

[23]  Cristopher Moore,et al.  Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[24]  Cristopher Moore,et al.  Model selection for degree-corrected block models , 2012, Journal of statistical mechanics.