Synergy, Redundancy, and Independence in Population Codes

A key issue in understanding the neural code for an ensemble of neurons is the nature and strength of correlations between neurons and how these correlations are related to the stimulus. The issue is complicated by the fact that there is not a single notion of independence or lack of correlation. We distinguish three kinds: (1) activity independence; (2) conditional independence; and (3) information independence. Each notion is related to an information measure: the information between cells, the information between cells given the stimulus, and the synergy of cells about the stimulus, respectively. We show that these measures form an interrelated framework for evaluating contributions of signal and noise correlations to the joint information conveyed about the stimulus and that at least two of the three measures must be calculated to characterize a population code. This framework is compared with others recently proposed in the literature. In addition, we distinguish questions about how information is encoded by a population of neurons from how that information can be decoded. Although information theory is natural and powerful for questions of encoding, it is not sufficient for characterizing the process of decoding. Decoding fundamentally requires an error measure that quantifies the importance of the deviations of estimated stimuli from actual stimuli. Because there is no a priori choice of error measure, questions about decoding cannot be put on the same level of generality as for encoding.

[1]  Liam Paninski,et al.  Estimation of Entropy and Mutual Information , 2003, Neural Computation.

[2]  W. McCulloch,et al.  The limiting information capacity of a neuronal link , 1952 .

[3]  William Bialek,et al.  Synergy in a Neural Code , 2000, Neural Computation.

[4]  Stefano Panzeri,et al.  A Unified Approach to the Study of Temporal, Correlational, and Rate Coding , 1999, Neural Computation.

[5]  M. Diamond,et al.  Population Coding of Stimulus Location in Rat Somatosensory Cortex , 2001, Neuron.

[6]  T. Shibasaki,et al.  Retinal ganglion cells act largely as independent encoders , 2001 .

[7]  R. Reid,et al.  Temporal Coding of Visual Information in the Thalamus , 2000, The Journal of Neuroscience.

[8]  M K Habib,et al.  Dynamics of neuronal firing correlation: modulation of "effective connectivity". , 1989, Journal of neurophysiology.

[9]  William Bialek,et al.  Real-time performance of a movement-sensitive neuron in the blowfly visual system: coding and information transfer in short spike sequences , 1988, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[10]  R. Reid,et al.  Precisely correlated firing in cells of the lateral geniculate nucleus , 1996, Nature.

[11]  Pamela Reinagel,et al.  Decoding visual information from a population of retinal ganglion cells. , 1997, Journal of neurophysiology.

[12]  Christof Koch,et al.  Stimulus Encoding and Feature Extraction by Multiple Sensory Neurons , 2002, The Journal of Neuroscience.

[13]  A. Aertsen,et al.  On the significance of correlations among neuronal spike trains , 2004, Biological Cybernetics.

[14]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[15]  Ehud Zohary,et al.  Correlated neuronal discharge rate and its implications for psychophysical performance , 1994, Nature.

[16]  Reinhard Eckhorn,et al.  Rigorous and extended application of information theory to the afferent visual system of the cat. I. Basic concepts , 2004, Kybernetik.

[17]  Naftali Tishby,et al.  Synergy and Redundancy among Brain Cells of Behaving Monkeys , 1998, NIPS.

[18]  Gal Chechik,et al.  Group Redundancy Measures Reveal Redundancy Reduction in the Auditory Pathway , 2001, NIPS.

[19]  G. Laurent,et al.  Encoding of Olfactory Information with Oscillating Neural Assemblies , 1994, Science.

[20]  L. Paninski,et al.  Information about movement direction obtained from synchronous activity of motor cortical neurons. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[21]  A. Dawid.  Conditional Independence in Statistical Theory , 1979 .

[22]  F. Downton,et al.  Introduction to Mathematical Statistics , 1959 .

[23]  Peter Dayan,et al.  The Effect of Correlated Variability on the Accuracy of a Population Code , 1999, Neural Computation.

[24]  J. Victor.  Binless strategies for estimation of information from neural data. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[25]  Y. Dan,et al.  Coding of visual information by precisely correlated spikes in the lateral geniculate nucleus , 1998, Nature Neuroscience.

[26]  G D Lewen,et al.  Neural coding of naturalistic motion stimuli , 2001, Network.

[27]  Aapo Hyvärinen,et al.  A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images , 2001, Vision Research.

[28]  A. Aertsen,et al.  Dynamics of neuronal interactions in monkey cortex in relation to behavioural events , 1995, Nature.

[29]  Michael J. Berry,et al.  Refractoriness and Neural Precision , 1997, The Journal of Neuroscience.

[30]  Michele Bezzi,et al.  Redundancy and Synergy Arising from Pairwise Correlations in Neuronal Ensembles , 2002, Journal of Computational Neuroscience.

[31]  Carlos D. Brody,et al.  Correlations Without Synchrony , 1999, Neural Computation.

[32]  H Barlow,et al.  Redundancy reduction revisited , 2001, Network.

[33]  Eero P. Simoncelli,et al.  Natural signal statistics and sensory gain control , 2001, Nature Neuroscience.

[34]  Sheila Nirenberg,et al.  Decoding neuronal spike trains: How important are correlations? , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[35]  H. Sompolinsky,et al.  Population coding in neuronal systems with correlated noise. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[36]  L. Optican,et al.  Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. III. Information theoretic analysis. , 1987, Journal of neurophysiology.

[37]  D. Mastronarde.  Correlated firing of cat retinal ganglion cells. I. Spontaneously active inputs to X- and Y-cells. , 1983, Journal of neurophysiology.

[38]  Michael J. Berry,et al.  Network information and connected correlations. , 2003, Physical review letters.

[39]  F. Attneave.  Some informational aspects of visual perception. , 1954, Psychological review.

[40]  H. Sompolinsky,et al.  Mutual information of population codes and distance measures in probability space. , 2001, Physical review letters.

[41]  Michael DeWeese,et al.  Optimization Principles for the Neural Code , 1995, NIPS.

[42]  Alexandre Pouget,et al.  Probabilistic Interpretation of Population Codes , 1996, Neural Computation.

[43]  B L McNaughton,et al.  Interpreting neuronal population activity by reconstruction: unified framework with application to hippocampal place cells. , 1998, Journal of neurophysiology.

[44]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[45]  A. P. Georgopoulos,et al.  Neuronal population coding of movement direction. , 1986, Science.

[46]  Don H. Johnson,et al.  Information-Theoretic Analysis of Neural Coding , 2004, Journal of Computational Neuroscience.

[47]  Reinhard Eckhorn,et al.  Rigorous and extended application of information theory to the afferent visual system of the cat , 2004, Biological Cybernetics.

[48]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .

[49]  William Bialek,et al.  Entropy and Information in Neural Spike Trains , 1996, cond-mat/9603127.

[50]  William Bialek,et al.  Entropy and information in neural spike trains: progress on the sampling problem. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[51]  S. Laughlin,et al.  Predictive coding: a fresh view of inhibition in the retina , 1982, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[52]  M. Abeles,et al.  Transmission of information by the axon: II. The channel capacity , 2004, Biological Cybernetics.

[53]  William Bialek,et al.  Reading a Neural Code , 1991, NIPS.

[54]  Jianhua Lin,et al.  Divergence measures based on the Shannon entropy , 1991, IEEE Trans. Inf. Theory.

[55]  E. Vaadia,et al.  Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. , 1993, Journal of neurophysiology.

[56]  TJ Gawne,et al.  How independent are the messages carried by adjacent inferior temporal cortical neurons? , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[57]  W. Singer,et al.  Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[58]  J. V. van Hateren,et al.  Real and optimal neural images in early vision , 1992, Nature.

[59]  Inés Samengo,et al.  Information Loss in an Optimal Maximum Likelihood Decoding , 2001, Neural Computation.

[60]  D. Ruderman.  The statistics of natural images , 1994 .

[61]  S. Panzeri,et al.  An exact method to quantify the information transmitted by different mechanisms of correlational coding. , 2003, Network.

[62]  Si Wu,et al.  Sequential Bayesian Decoding with a Population of Neurons , 2003, Neural Computation.

[63]  F. Mechler,et al.  Interspike Intervals, Receptive Fields, and Information Encoding in Primary Visual Cortex , 2000, The Journal of Neuroscience.

[64]  D. Baylor,et al.  Concerted Signaling by Retinal Ganglion Cells , 1995, Science.

[65]  H. E. Derksen,et al.  Fluctuation phenomena in nerve membrane , 1968 .

[66]  Michael J. Berry,et al.  An Information Theoretic Approach to the Functional Classification of Neurons , 2002, NIPS.

[67]  Stefano Panzeri,et al.  The Upward Bias in Measures of Information Derived from Limited Data Samples , 1995, Neural Computation.

[68]  E T Rolls,et al.  Correlations and the encoding of information in the nervous system , 1999, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[69]  G. P. Moore,et al.  Neuronal spike trains and stochastic point processes. I. The single spike train. , 1967, Biophysical journal.

[70]  Naftali Tishby,et al.  Universality and Individuality in a Neural Code , 2000, NIPS.

[71]  Robert V. Hogg,et al.  Introduction to Mathematical Statistics. , 1966 .

[72]  Nicolas Brunel,et al.  Mutual Information, Fisher Information, and Population Coding , 1998, Neural Computation.

[73]  H Sompolinsky,et al.  Simple models for reading neuronal population codes. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[74]  Emilio Salinas,et al.  Vector reconstruction from firing rates , 1994, Journal of Computational Neuroscience.

[75]  T. Albright,et al.  Efficient Discrimination of Temporal Patterns by Motion-Sensitive Neurons in Primate Visual Cortex , 1998, Neuron.

[76]  J. H. van Hateren,et al.  Real and optimal neural images in early vision , 1992, Nature.

[77]  Christof Koch,et al.  Temporal Precision of Spike Trains in Extrastriate Cortex of the Behaving Macaque Monkey , 1999, Neural Computation.

[78]  Richard FrrZHUGH.  THE STATISTICAL DETECTION OF THRESHOLD SIGNALS IN THE RETINA , 1957 .

[79]  E N Brown,et al.  A Statistical Paradigm for Neural Spike Train Decoding Applied to Position Prediction from Ensemble Firing Patterns of Rat Hippocampal Place Cells , 1998, The Journal of Neuroscience.

[80]  O. Prospero-Garcia,et al.  Reliability of Spike Timing in Neocortical Neurons , 1995 .

[81]  G. P. Moore,et al.  Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains. , 1967, Biophysical journal.

[82]  Kathryn B. Laskey,et al.  Neural Coding: Higher-Order Temporal Patterns in the Neurostatistics of Cell Assemblies , 2000, Neural Computation.