Spectral estimation for detecting low-dimensional structure in networks using arbitrary null models

Discovering low-dimensional structure in real-world networks requires a suitable null model that defines the absence of meaningful structure. Here we introduce a spectral approach for detecting a network's low-dimensional structure, and the nodes that participate in it, using any null model. We use generative models to estimate the expected eigenvalue distribution under a specified null model, and then detect where the data network's eigenspectra exceed the estimated bounds. On synthetic networks, this spectral estimation approach cleanly detects transitions between random and community structure, recovers the number and membership of communities, and removes noise nodes. On real networks spectral estimation finds either a significant fraction of noise nodes or no departure from a null model, in stark contrast to traditional community detection methods. Across all analyses, we find the choice of null model can strongly alter conclusions about the presence of network structure. Our spectral estimation approach is therefore a promising basis for detecting low-dimensional structure in real-world networks, or lack thereof.

[1]  Joel Nishimura,et al.  Configuring Random Graph Models with Fixed Degree Sequences , 2016, SIAM Rev..

[2]  Mark D. Humphries,et al.  Finding communities in sparse networks , 2015, Scientific Reports.

[3]  Aaron Clauset,et al.  Evaluating Overfit and Underfit in Models of Network Community Structure , 2018, IEEE Transactions on Knowledge and Data Engineering.

[4]  E. Ott,et al.  Spectral properties of networks with community structure. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[5]  Cristopher Moore,et al.  Scalable detection of statistically significant communities and hierarchies, using message passing for modularity , 2014, Proceedings of the National Academy of Sciences.

[6]  Lydia Ng,et al.  Clustering of spatial gene expression patterns in the mouse brain and comparison with classical neuroanatomy. , 2010, Methods.

[7]  Aaron Clauset,et al.  Learning Latent Block Structure in Weighted Networks , 2014, J. Complex Networks.

[8]  Adriano B. L. Tort,et al.  Neuronal Assembly Detection and Cell Membership Specification by Principal Component Analysis , 2011, PloS one.

[9]  Mark E. J. Newman,et al.  Spectral community detection in sparse networks , 2013, ArXiv.

[10]  Cristopher Moore,et al.  Community detection in networks with unequal groups , 2015, Physical review. E.

[11]  Mark E. J. Newman,et al.  Stochastic blockmodels and community structure in networks , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[12]  Mark D. Humphries,et al.  Modular Deconstruction Reveals the Dynamical and Physical Building Blocks of a Locomotion Motor Program , 2015, Neuron.

[13]  Benjamin H. Good,et al.  Performance of modularity maximization in practical contexts. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[14]  Danielle S Bassett,et al.  Generative models for network neuroscience: prospects and promise , 2017, Journal of The Royal Society Interface.

[15]  Olaf Sporns,et al.  Weight-conserving characterization of complex functional brain networks , 2011, NeuroImage.

[16]  Santo Fortunato,et al.  Community detection in networks: A user guide , 2016, ArXiv.

[17]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[18]  P. Good Permutation, Parametric, and Bootstrap Tests of Hypotheses , 2005 .

[19]  Christopher M. Danforth,et al.  Estimation of Global Network Statistics from Incomplete Data , 2014, PloS one.

[20]  David Lusseau,et al.  The emergent properties of a dolphin social network , 2003, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[21]  M. Newman,et al.  Finding community structure in networks using the eigenvectors of matrices. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[22]  Naoki Masuda,et al.  Configuration model for correlation matrices preserving the node strength. , 2018, Physical review. E.

[23]  Tiago P. Peixoto Nonparametric weighted stochastic block models. , 2017, Physical review. E.

[24]  Jean-Gabriel Young,et al.  Construction of and efficient sampling from the simplicial configuration model. , 2017, Physical review. E.

[25]  Jure Leskovec,et al.  Defining and evaluating network communities based on ground-truth , 2012, KDD 2012.

[26]  D. Garlaschelli,et al.  Community detection for correlation matrices , 2013, 1311.1924.

[27]  K. Gurney,et al.  Network ‘Small-World-Ness’: A Quantitative Method for Determining Canonical Network Equivalence , 2008, PloS one.

[28]  Leto Peel,et al.  The ground truth about metadata and community detection in networks , 2016, Science Advances.

[29]  V. Plerou,et al.  Random matrix approach to cross correlations in financial data. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[30]  Mark D Humphries,et al.  Spike-Train Communities: Finding Groups of Similar Spike Trains , 2011, The Journal of Neuroscience.

[31]  M. Meilă Comparing clusterings---an information based distance , 2007 .

[32]  Andrew B. Nobel,et al.  Significance-based community detection in weighted networks , 2016, J. Mach. Learn. Res..

[33]  Elchanan Mossel,et al.  Spectral redemption in clustering sparse networks , 2013, Proceedings of the National Academy of Sciences.

[34]  Jean-Loup Guillaume,et al.  Fast unfolding of communities in large networks , 2008, 0803.0476.

[35]  Santo Fortunato,et al.  Improving the performance of algorithms to find communities in networks , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[36]  S. Fortunato,et al.  Resolution limit in community detection , 2006, Proceedings of the National Academy of Sciences.

[37]  Mehdi Khamassi,et al.  Principal component analysis of ensemble recordings reveals cell assemblies at high temporal resolution , 2009, Journal of Computational Neuroscience.

[38]  Mason A. Porter,et al.  Core-Periphery Structure in Networks , 2012, SIAM J. Appl. Math..

[39]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[40]  A. D. Medus,et al.  Community Detection in Networks , 2010, Int. J. Bifurc. Chaos.

[41]  J. Reichardt,et al.  Statistical mechanics of community detection. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[42]  Xiao Zhang,et al.  Spectra of random graphs with community structure and arbitrary degrees , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[43]  Allan R. Jones,et al.  An anatomic gene expression atlas of the adult mouse brain , 2009, Nature Neuroscience.

[44]  Tiago P. Peixoto Model selection and hypothesis testing for large-scale network models with overlapping groups , 2014, ArXiv.

[45]  Mark E. J. Newman,et al.  The Structure and Function of Complex Networks , 2003, SIAM Rev..

[46]  M. Newman Analysis of weighted networks. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[47]  M E J Newman,et al.  Finding and evaluating community structure in networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.