Sparse low-order interaction network underlies a highly correlated and learnable neural population code

Information is carried in the brain by the joint activity patterns of large groups of neurons. Understanding the structure and function of population neural codes is challenging because of the exponential number of possible activity patterns and dependencies among neurons. We report here that for groups of ~100 retinal neurons responding to natural stimuli, pairwise-based models, which were highly accurate for small networks, are no longer sufficient. We show that because of the sparse nature of the neural code, the higher-order interactions can be easily learned using a novel model and that a very sparse low-order interaction network underlies the code of large populations of neurons. Additionally, we show that the interaction network is organized in a hierarchical and modular manner, which hints at scalability. Our results suggest that learnability may be a key feature of the neural code.

[1]  E. Jaynes Information Theory and Statistical Mechanics , 1957 .

[2]  J. Besag Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[3]  J. Besag Efficiency of pseudolikelihood estimation for simple Gaussian fields , 1977 .

[4]  J. Hopfield,et al.  Computing with neural circuits: a model. , 1986, Science.

[5]  A. P. Georgopoulos,et al.  Neuronal population coding of movement direction. , 1986, Science.

[6]  Wang,et al.  Nonuniversal critical dynamics in Monte Carlo simulations. , 1987, Physical review letters.

[7]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[8]  Wolff,et al.  Collective Monte Carlo updating for spin systems. , 1989, Physical review letters.

[9]  E. Marder,et al.  Activity-dependent regulation of conductances in model neurons. , 1993, Science.

[10]  Gerasimos Potamianos,et al.  Partition function estimation of Gibbs random field images using Monte Carlo simulations , 1993, IEEE Trans. Inf. Theory.

[11]  E. Marder,et al.  Activity-dependent changes in the intrinsic properties of cultured neurons. , 1994, Science.

[12]  Markus Meister,et al.  Multi-neuronal signals from the retina: acquisition and analysis , 1994, Journal of Neuroscience Methods.

[13]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .

[14]  Gerasimos Potamianos,et al.  Stochastic approximation algorithms for partition function estimation of Gibbs random fields , 1997, IEEE Trans. Inf. Theory.

[15]  G. Laurent,et al.  Impaired odour discrimination on desynchronization of odour-encoding neural assemblies , 1997, Nature.

[16]  A. Aertsen,et al.  Spike synchronization and rate modulation differentially involved in motor cortical function. , 1997, Science.

[17]  L. Paninski,et al.  Information about movement direction obtained from synchronous activity of motor cortical neurons. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[18]  E N Brown,et al.  A Statistical Paradigm for Neural Spike Train Decoding Applied to Position Prediction from Ensemble Firing Patterns of Rat Hippocampal Place Cells , 1998, The Journal of Neuroscience.

[19]  Y. Dan,et al.  Coding of visual information by precisely correlated spikes in the lateral geniculate nucleus , 1998, Nature Neuroscience.

[20]  Kathryn B. Laskey,et al.  Neural Coding: Higher-Order Temporal Patterns in the Neurostatistics of Cell Assemblies , 2000, Neural Computation.

[21]  H Barlow,et al.  Redundancy reduction revisited , 2001, Network.

[22]  W. Bair,et al.  Correlated Firing in Macaque Visual Area MT: Time Scales and Relationship to Behavior , 2001, The Journal of Neuroscience.

[23]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[24]  Shun-ichi Amari,et al.  Information geometry on hierarchy of probability distributions , 2001, IEEE Trans. Inf. Theory.

[25]  Michael J. Berry,et al.  Synergy, Redundancy, and Independence in Population Codes , 2003, The Journal of Neuroscience.

[26]  M. Schnitzer,et al.  Multineuronal Firing Patterns in the Signal from Eye to Brain , 2003, Neuron.

[27]  Michael J. Berry,et al.  Network information and connected correlations. , 2003, Physical review letters.

[28]  J. Csicsvari,et al.  Organization of cell assemblies in the hippocampus , 2003, Nature.

[29]  Bruno A Olshausen,et al.  Sparse coding of sensory inputs , 2004, Current Opinion in Neurobiology.

[30]  Daeyeol Lee,et al.  Coding and transmission of information by neural ensembles , 2004, Trends in Neurosciences.

[31]  Michael J. Berry,et al.  Redundancy in the Population Code of the Retina , 2005, Neuron.

[32]  M. Laubach,et al.  Redundancy and Synergy of Neuronal Ensembles in Motor Cortex , 2005, The Journal of Neuroscience.

[33]  Michael J. Berry,et al.  Ising models for networks of real neurons , 2006, q-bio/0611072.

[34]  Michael J. Berry,et al.  Weak pairwise correlations imply strongly correlated network states in a neural population , 2005, Nature.

[35]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[36]  Jonathon Shlens,et al.  The Structure of Multi-Neuron Firing Patterns in Primate Retina , 2006, The Journal of Neuroscience.

[37]  Gal Chechik,et al.  Reduction of Information Redundancy in the Ascending Auditory Pathway , 2006, Neuron.

[38]  Pieter Abbeel,et al.  Learning Factor Graphs in Polynomial Time and Sample Complexity , 2006, J. Mach. Learn. Res..

[39]  Michael J. Berry,et al.  Role of eye movements in the retinal code for a size discrimination task. , 2007, Journal of neurophysiology.

[40]  Matthias Bethge,et al.  Near-Maximum Entropy Models for Binary Neural Representations of Natural Images , 2007, NIPS.

[41]  Eero P. Simoncelli,et al.  Spatio-temporal correlations and visual signalling in a complete neuronal population , 2008, Nature.

[42]  K. Svoboda,et al.  Sparse optical microstimulation in barrel cortex drives learned behaviour in freely moving mice , 2008, Nature.

[43]  Robert E. Schapire,et al.  Faster solutions of the inverse pairwise Ising problem , 2008 .

[44]  John M. Beggs,et al.  A Maximum Entropy Model Applied to Spatial and Temporal Correlations from Cortical Networks In Vitro , 2008, The Journal of Neuroscience.

[45]  Jonathon Shlens,et al.  The Structure of Large-Scale Synchronized Firing in Primate Retina , 2009, The Journal of Neuroscience.

[46]  Sami El Boustani,et al.  Prediction of spatiotemporal patterns of neural activity from pairwise correlations. , 2009, Physical review letters.

[47]  S. Leibler,et al.  Neuronal couplings between retinal ganglion cells inferred by efficient inverse statistical physics methods , 2009, Proceedings of the National Academy of Sciences.

[48]  Ifije E. Ohiorhenuan,et al.  Sparse coding and high-order correlations in fine-scale cortical networks , 2010, Nature.

[49]  Dietmar Plenz,et al.  Hierarchical Interaction Structure of Neural Activities in Cortical Slice Cultures , 2010, The Journal of Neuroscience.

[50]  Radford M. Neal Probabilistic Inference Using Markov Chain Monte Carlo Methods , 2011 .

[51]  R. Segev,et al.  The Architecture of Functional Interaction Networks in the Retina , 2011, The Journal of Neuroscience.