Network information and connected correlations.

Entropy and information provide natural measures of correlation among elements in a network. We construct here the information theoretic analog of connected correlation functions: irreducible N-point correlation is measured by a decrease in entropy for the joint distribution of N variables relative to the maximum entropy allowed by all the observed N-1 variable distributions. We calculate the "connected information" terms for several examples and show that it also enables the decomposition of the information that is carried by a population of elements about an outside source.

[1]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[2]  William J. McGill.  Multivariate information transmission , 1954, Trans. IRE Prof. Group Inf. Theory.

[3]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[4]  Michael Satosi Watanabe,et al.  Information Theoretical Analysis of Multivariate Correlation , 1960, IBM J. Res. Dev..

[5]  I. Good.  Maximum Entropy for Hypothesis Formulation, Especially for Multidimensional Contingency Tables , 1963 .

[6]  I. Csiszár.  $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[7]  Ehsan S. Soofi,et al.  A Generalizable Formulation of Conditional Logit with Diagnostics , 1992 .

[8]  D. Vernon.  Inform , 1995, Encyclopedia of the UN Sustainable Development Goals.

[9]  Kathryn B. Laskey,et al.  Neural Coding: Higher-Order Temporal Patterns in the Neurostatistics of Cell Assemblies , 2000, Neural Computation.

[10]  William Bialek,et al.  Synergy in a Neural Code , 2000, Neural Computation.

[11]  D. Wilkin,et al.  Neuron , 2001, Brain Research.

[12]  Shun-ichi Amari,et al.  Information geometry on hierarchy of probability distributions , 2001, IEEE Trans. Inf. Theory.

[13]  W. Wootters,et al.  Almost every pure state of three qubits is completely determined by its two-particle reduced density matrices. , 2002, Physical review letters.

[14]  M. Schnitzer,et al.  Multineuronal Firing Patterns in the Signal from Eye to Brain , 2003, Neuron.

[15]  A. Aertsen,et al.  On the significance of correlations among neuronal spike trains , 2004, Biological Cybernetics.

[16]  Ericka Stricklin-Parker,et al.  Ann , 2005 .