Reducing degeneracy in maximum entropy models of networks.

Based on Jaynes's maximum entropy principle, exponential random graphs provide a family of principled models that allow the prediction of network properties as constrained by empirical data (observables). However, their use is often hindered by the degeneracy problem characterized by spontaneous symmetry breaking, where predictions fail. Here we show that degeneracy appears when the corresponding density of states function is not log-concave, which is typically the consequence of nonlinear relationships between the constraining observables. Exploiting these nonlinear relationships here we propose a solution to the degeneracy problem for a large class of systems via transformations that render the density of states function log-concave. The effectiveness of the method is demonstrated on examples.

[1]  P. Pattison,et al.  New Specifications for Exponential Random Graph Models , 2006 .

[2]  István Miklós,et al.  On realizations of a joint degree matrix , 2015, Discret. Appl. Math..

[3]  Hyunju Kim,et al.  Degree-based graph construction , 2009, 0905.4892.

[4]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[5]  P. Holland,et al.  An Exponential Family of Probability Distributions for Directed Graphs , 1981 .

[6]  Kevin Lewis,et al.  Beyond and Below Racial Homophily: ERG Models of a Friendship Network Documented on Facebook1 , 2010, American Journal of Sociology.

[7]  宁北芳,et al.  疟原虫var基因转换速率变化导致抗原变异[英]/Paul H, Robert P, Christodoulou Z, et al//Proc Natl Acad Sci U S A , 2005 .

[8]  Alan M. Frieze,et al.  Random graphs , 2006, SODA '06.

[9]  Robert P. Anderson,et al.  Maximum entropy modeling of species geographic distributions , 2006 .

[10]  M. Newman,et al.  Statistical mechanics of networks. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[11]  김삼묘,et al.  “Bioinformatics” 특집을 내면서 , 2000 .

[12]  William Bialek,et al.  Searching for simplicity in the analysis of neurons and behavior , 2010, Proceedings of the National Academy of Sciences.

[13]  K. Dill,et al.  Principles of maximum entropy and maximum caliber in statistical physics , 2013 .

[14]  Nicholas Economides,et al.  Internet and Network Economics , 2011, Lecture Notes in Computer Science.

[15]  R. Rosenfeld Nature , 2009, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[16]  J. Brochon,et al.  Analyzing the distribution of decay constants in pulse-fluorimetry using the maximum entropy method. , 1987, Biophysical journal.

[17]  Naoki Masuda,et al.  A pairwise maximum entropy model accurately describes resting-state human brain networks , 2013, Nature Communications.

[18]  Garry Robins,et al.  An introduction to exponential random graph (p*) models for social networks , 2007, Soc. Networks.

[19]  Aleksandra M. Walczak,et al.  Optimizing Information Flow in Small Genetic Networks. II. Feed-forward Interactions , 2010 .

[20]  Christopher B. Burge,et al.  Maximum Entropy Modeling of Short Sequence Motifs with Applications to RNA Splicing Signals , 2004, J. Comput. Biol..

[21]  G Bricogne,et al.  Maximum-entropy three-dimensional reconstruction with deconvolution of the contrast transfer function: a test application with adenovirus. , 1996, Journal of structural biology.

[22]  S. Gull,et al.  Image reconstruction from incomplete and noisy data , 1978, Nature.

[23]  Peter G Wolynes,et al.  Coevolutionary information, protein folding landscapes, and the thermodynamics of natural selection , 2014, Proceedings of the National Academy of Sciences.

[24]  Pablo M. Gleiser,et al.  Community Structure in Jazz , 2003, Adv. Complex Syst..

[25]  Massimo Vergassola,et al.  ‘Infotaxis’ as a strategy for searching without gradients , 2007, Nature.

[26]  Juyong Park,et al.  Solution for the properties of a clustered network. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[27]  Sami El Boustani,et al.  Prediction of spatiotemporal patterns of neural activity from pairwise correlations. , 2009, Physical review letters.

[28]  Remo Guidieri Res , 1995, RES: Anthropology and Aesthetics.

[29]  M. Newman,et al.  Solution of the two-star model of a network. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[30]  Agata Fronczak,et al.  Exponential random graph models for networks with community structure , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[31]  R. Segev,et al.  Sparse low-order interaction network underlies a highly correlated and learnable neural population code , 2011, Proceedings of the National Academy of Sciences.

[32]  Béla Bollobás,et al.  Random Graphs: Notation , 2001 .

[33]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[34]  R. Gilmore,et al.  Group Theory , 2010 .

[35]  W. Zachary,et al.  An Information Flow Model for Conflict and Fission in Small Groups , 1977, Journal of Anthropological Research.

[36]  Thomas A. House Heterogeneous clustered random graphs , 2014 .

[37]  D. Wilkin,et al.  Neuron , 2001, Brain Research.

[38]  Aonan Tang,et al.  Maximum Entropy Approaches to Living Neural Networks , 2010, Entropy.

[39]  Ronald Rosenfeld,et al.  A maximum entropy approach to adaptive statistical language modelling , 1996, Comput. Speech Lang..

[40]  Kevin E. Bassler,et al.  Efficient and Exact Sampling of Simple Graphs with Given Arbitrary Degree Sequence , 2010, PloS one.

[41]  Henry Kennedy,et al.  A Predictive Network Model of Cerebral Cortical Connectivity Based on a Distance Rule , 2013, Neuron.

[42]  Michael J. Berry,et al.  Weak pairwise correlations imply strongly correlated network states in a neural population , 2005, Nature.

[43]  David Strauss On a general class of models for interaction , 1986 .

[44]  Haw Yang,et al.  Quantitative single-molecule conformational distributions: a case study with poly-(L-proline). , 2006, The journal of physical chemistry. A.

[45]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[46]  Jonathon Shlens,et al.  The Structure of Multi-Neuron Firing Patterns in Primate Retina , 2006, The Journal of Neuroscience.

[47]  John M. Beggs,et al.  A Maximum Entropy Model Applied to Spatial and Temporal Correlations from Cortical Networks In Vitro , 2008, The Journal of Neuroscience.

[48]  Agata Fronczak,et al.  Analysis of scientific productivity using maximum entropy principle and fluctuation-dissipation theorem. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[49]  C. Matthews,et al.  Analysis of kinetics using a hybrid maximum-entropy/nonlinear-least-squares method: application to protein folding. , 2002, Biophysical journal.

[50]  Leslie G. Valiant,et al.  Random Generation of Combinatorial Structures from a Uniform Distribution , 1986, Theor. Comput. Sci..

[51]  Vijay V. Vazirani,et al.  Approximation Algorithms , 2001, Springer Berlin Heidelberg.

[52]  E. Jaynes Information Theory and Statistical Mechanics , 1957 .

[53]  Vladimir Filkov,et al.  Exploring biological network structure using exponential random graph models , 2007, Bioinform..