Efficient associative memory storage in cortical circuits of inhibitory and excitatory neurons

Many features of synaptic connectivity are ubiquitous among cortical systems. Cortical networks are dominated by excitatory neurons and synapses, are sparsely connected, and function with stereotypically distributed connection weights. We show that these basic structural and functional features of synaptic connectivity arise readily from the requirement of efficient associative memory storage. Our theory makes two fundamental predictions. First, we predict that, despite a large number of neuron classes, functional connections between potentially connected cells must be realized with <50% probability if the presynaptic cell is excitatory and >50% probability if the presynaptic cell is inhibitory. Second, we establish a unique relation between probability of connection and coefficient of variation in connection weights. These predictions are consistent with a dataset of 74 published experiments reporting connection probabilities and distributions of postsynaptic potential amplitudes in various cortical systems. What is more, our theory explains the shapes of the distributions obtained in these experiments.

[1]  Rafael Yuste,et al.  Ultrastructure of Dendritic Spines: Correlation Between Synaptic and Spine Morphologies , 2007, Front. Neurosci..

[2]  T. Bliss,et al.  Long‐lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path , 1973, The Journal of physiology.

[3]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[4]  Karel Svoboda,et al.  Rapid Functional Maturation of Nascent Dendritic Spines , 2009, Neuron.

[5]  E. Gardner The space of interactions in neural network models , 1988 .

[6]  Raju R. Viswanathan,et al.  Sign-constrained synapses and biased patterns in neural networks , 1993 .

[7]  C. Gray,et al.  Physiological properties of inhibitory interneurons in cat striate cortex. , 1997, Cerebral cortex.

[8]  Q. Ouyang,et al.  The yeast cell-cycle network is robustly designed. , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[9]  B. Sakmann,et al.  The Excitatory Neuronal Network of Rat Layer 4 Barrel Cortex , 2000, The Journal of Neuroscience.

[10]  Hongkui Zeng,et al.  Differential tuning and population dynamics of excitatory and inhibitory neurons reflect differences in local intracortical connectivity , 2011, Nature Neuroscience.

[11]  D. Chklovskii,et al.  Neurogeometry and potential synaptic connectivity , 2005, Trends in Neurosciences.

[12]  Mu-ming Poo,et al.  Shrinkage of Dendritic Spines Associated with Long-Term Depression of Hippocampal Synapses , 2004, Neuron.

[13]  Bartlett W. Mel,et al.  Cortical rewiring and information storage , 2004, Nature.

[14]  C. Holmgren,et al.  Coincident Spiking Activity Induces Long-Term Changes in Inhibition of Neocortical Pyramidal Cells , 2001, The Journal of Neuroscience.

[15]  C. Petersen,et al.  The Excitatory Neuronal Network of the C2 Barrel Column in Mouse Primary Somatosensory Cortex , 2009, Neuron.

[16]  Chris I. De Zeeuw,et al.  Elimination of Inhibitory Synapses Is a Major Component of Adult Ocular Dominance Plasticity , 2012, Neuron.

[17]  Prof. Dr. Dr. Valentino Braitenberg,et al.  Cortex: Statistics and Geometry of Neuronal Connectivity , 1998, Springer Berlin Heidelberg.

[18]  J. Nadal,et al.  What can we learn from synaptic weight distributions? , 2007, Trends in Neurosciences.

[19]  Moritz Helmstaedter,et al.  Efficient Recruitment of Layer 2/3 Interneurons by Layer 4 Input in Single Columns of Rat Somatosensory Cortex , 2008, The Journal of Neuroscience.

[20]  J. Nadal,et al.  Optimal Information Storage and the Distribution of Synaptic Weights Perceptron versus Purkinje Cell , 2004, Neuron.

[21]  Sen Song,et al.  Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits , 2005, PLoS biology.

[22]  J. Lübke,et al.  Excitatory signal flow and connectivity in a cortical column: focus on barrel cortex , 2007, Brain Structure and Function.

[23]  N. Seidah,et al.  Regulation by gastric acid of the processing of progastrin‐derived peptides in rat antral mucosa , 1997, The Journal of physiology.

[24]  A. Thomson,et al.  Selectivity in the inter-laminar connections made by neocortical neurones , 2002, Journal of neurocytology.

[25]  M Rosenblatt,et al.  THE MULTIDIMENSIONAL PREDICTION PROBLEM. , 1957, Proceedings of the National Academy of Sciences of the United States of America.

[26]  D. Chklovskii,et al.  Geometry and Structural Plasticity of Synaptic Connectivity , 2002, Neuron.

[27]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[28]  H. Markram,et al.  Physiology and anatomy of synaptic connections between thick tufted pyramidal neurones in the developing rat neocortex. , 1997, The Journal of physiology.

[29]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[30]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[31]  R. Yuste,et al.  Dense Inhibitory Connectivity in Neocortex , 2011, Neuron.

[32]  Y. Kubota,et al.  Structural basis for the role of inhibition in facilitating adult brain plasticity , 2011, Nature Neuroscience.

[33]  J. Lübke,et al.  Reliable synaptic connections between pairs of excitatory layer 4 neurones within a single ‘barrel’ of developing rat somatosensory cortex , 1999, The Journal of physiology.

[34]  Stuart A. Kauffman,et al.  The origins of order , 1993 .

[35]  Alex S. Ferecskó,et al.  The fractions of short- and long-range connections in the visual cortex , 2009, Proceedings of the National Academy of Sciences.

[36]  Brent Doiron,et al.  Spatial Profile and Differential Recruitment of GABAB Modulate Oscillatory Activity in Auditory Cortex , 2009, The Journal of Neuroscience.

[37]  D. Hubel,et al.  Ferrier lecture - Functional architecture of macaque monkey visual cortex , 1977, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[38]  B. Barbour,et al.  Properties of Unitary Granule Cell→Purkinje Cell Synapses in Adult Rat Cerebellar Slices , 2002, The Journal of Neuroscience.

[39]  D. Marr A theory of cerebellar cortex , 1969, The Journal of physiology.

[40]  B. Connors,et al.  Two dynamically distinct inhibitory networks in layer 4 of the neocortex. , 2003, Journal of neurophysiology.

[41]  Thomas K. Berger,et al.  Brief Bursts Self-Inhibit and Correlate the Pyramidal Network , 2010, PLoS biology.

[42]  David A Lewis,et al.  Functional properties of fast spiking interneurons and their synaptic connections with pyramidal cells in primate dorsolateral prefrontal cortex. , 2005, Journal of neurophysiology.

[43]  K. Stratford,et al.  Synaptic transmission between individual pyramidal neurons of the rat visual cortex in vitro , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[44]  J. Eccles From electrical to chemical transmission in the central nervous system: The closing address of the Sir Henry Dale Centennial Symposium Cambridge, 19 September 1975 , 1976, Notes and Records of the Royal Society of London.

[45]  K. Svoboda,et al.  Long-term in vivo imaging of experience-dependent synaptic plasticity in adult cortex , 2002, Nature.

[46]  D. O. Hebb,et al.  The organization of behavior , 1988 .

[47]  C F Stevens,et al.  Quantitative fine-structural analysis of olfactory cortical synapses. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[48]  Jean-Pierre Nadal On the storage capacity with sign-constrained synaptic couplings , 1990 .

[49]  P. J. Sjöström,et al.  Rate, Timing, and Cooperativity Jointly Determine Cortical Synaptic Plasticity , 2001, Neuron.

[50]  R. Douglas,et al.  A Quantitative Map of the Circuit of Cat Primary Visual Cortex , 2004, The Journal of Neuroscience.

[51]  Yun Wang,et al.  Synaptic connections and small circuits involving excitatory and inhibitory neurons in layers 2-5 of adult rat and cat neocortex: triple intracellular recordings and biocytin labelling in vitro. , 2002, Cerebral cortex.

[52]  G. Ellis‐Davies,et al.  Structural basis of long-term potentiation in single dendritic spines , 2004, Nature.

[53]  J. Csicsvari,et al.  Oscillatory Coupling of Hippocampal Pyramidal Cells and Interneurons in the Behaving Rat , 1999, The Journal of Neuroscience.

[54]  E. Gardner,et al.  Optimal storage properties of neural network models , 1988 .

[55]  W. Newsome,et al.  The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding , 1998, The Journal of Neuroscience.

[56]  M. J. Friedlander,et al.  The time course and amplitude of EPSPs evoked at synapses between pairs of CA3/CA1 neurons in the hippocampal slice , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[57]  A. Stepanyants,et al.  Cooperative synapse formation in the neocortex , 2009, Proceedings of the National Academy of Sciences.

[58]  K. Harris,et al.  Dendritic spines of rat cerebellar Purkinje cells: serial electron microscopy with reference to their biophysical characteristics , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[59]  Karel Svoboda,et al.  Locally dynamic synaptic learning rules in pyramidal neuron dendrites , 2007, Nature.

[60]  Thomas K. Berger,et al.  Frequency‐dependent disynaptic inhibition in the pyramidal network: a ubiquitous pathway in the developing rat neocortex , 2009, The Journal of physiology.

[61]  A. Peters,et al.  Numerical relationships between geniculocortical afferents and pyramidal cell modules in cat primary visual cortex. , 1993, Cerebral cortex.

[62]  J. Knott The organization of behavior: A neuropsychological theory , 1951 .

[63]  D. Amit,et al.  Perceptron learning with sign-constrained weights , 1989 .

[64]  H. Markram,et al.  The neocortical microcircuit as a tabula rasa. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[65]  D. Hubel,et al.  Shape and arrangement of columns in cat's striate cortex , 1963, The Journal of physiology.

[66]  Elly Nedivi,et al.  Clustered Dynamics of Inhibitory Synapses and Dendritic Spines in the Adult Neocortex , 2012, Neuron.

[67]  John R Huguenard,et al.  Barrel Cortex Microcircuits: Thalamocortical Feedforward Inhibition in Spiny Stellate Cells Is Mediated by a Small Number of Fast-Spiking Interneurons , 2006, The Journal of Neuroscience.

[68]  A. Agmon,et al.  Submillisecond Firing Synchrony between Different Subtypes of Cortical Interneurons Connected Chemically But Not Electrically , 2011, The Journal of Neuroscience.

[69]  Alex S. Ferecskó,et al.  Local Potential Connectivity in Cat Primary Visual Cortex , 2008 .

[70]  Henry Markram,et al.  Deriving physical connectivity from neuronal morphology , 2003, Biological Cybernetics.

[71]  M. Frerking,et al.  Spine Expansion and Stabilization Associated with Long-Term Potentiation , 2008, The Journal of Neuroscience.

[72]  T. Wiesel,et al.  Functional architecture of macaque monkey visual cortex , 1977 .

[73]  Arantxa Etxeverria The Origins of Order , 1993 .

[74]  D J Amit,et al.  The interaction space of neural networks with sign-constrained synapses , 1989 .

[75]  R. Silver,et al.  Synaptic connections between layer 4 spiny neurone‐ layer 2/3 pyramidal cell pairs in juvenile rat barrel cortex: physiology and anatomy of interlaminar signalling within a cortical column , 2002, The Journal of physiology.

[76]  V. Mountcastle The columnar organization of the neocortex. , 1997, Brain : a journal of neurology.

[77]  H. Markram,et al.  Disynaptic Inhibition between Neocortical Pyramidal Cells Mediated by Martinotti Cells , 2007, Neuron.

[78]  P. Celada,et al.  In vivo modulation of the activity of pyramidal neurons in the rat medial prefrontal cortex by 5-HT2A receptors: relationship to thalamocortical afferents. , 2003, Cerebral cortex.

[79]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[80]  R. Yuste,et al.  Input Summation by Cultured Pyramidal Neurons Is Linear and Position-Independent , 1998, The Journal of Neuroscience.

[81]  R. Yuste,et al.  Linear Summation of Excitatory Inputs by CA1 Pyramidal Neurons , 1999, Neuron.

[82]  Tobias Bonhoeffer,et al.  Loss of Sensory Input Causes Rapid Structural Changes of Inhibitory Neurons in Adult Mouse Visual Cortex , 2011, Neuron.

[83]  D. Widmaier,et al.  Sign-constrained linear learning and diluting in neural networks , 1991 .

[84]  Roberto Malinow,et al.  Glutamate Receptor Exocytosis and Spine Enlargement during Chemically Induced Long-Term Potentiation , 2006, The Journal of Neuroscience.

[85]  Siu Kang,et al.  Bidirectional plasticity in fast-spiking GABA circuits by visual experience , 2009, Nature.

[86]  Maurice J. Chacron,et al.  Nonrenewal spike train statistics: causes and functional consequences on neural coding , 2011, Experimental Brain Research.

[87]  T. Harkany,et al.  Pyramidal cell communication within local networks in layer 2/3 of rat neocortex , 2003, The Journal of physiology.

[88]  T. Hromádka,et al.  Sparse Representation of Sounds in the Unanesthetized Auditory Cortex , 2008, PLoS biology.