Winnerless competition in clustered balanced networks: inhibitory assemblies do the trick

Balanced networks are a frequently employed basic model for neuronal networks in the mammalian neocortex. Large numbers of excitatory and inhibitory neurons are recurrently connected so that the numerous positive and negative inputs that each neuron receives cancel out on average. Neuronal firing is therefore driven by fluctuations in the input and resembles the irregular and asynchronous activity observed in cortical in vivo data. Recently, the balanced network model has been extended to accommodate clusters of strongly interconnected excitatory neurons in order to explain persistent activity in working memory-related tasks. This clustered topology introduces multistability and winnerless competition between attractors and can capture the high trial-to-trial variability and its reduction during stimulation that has been found experimentally. In this prospect article, we review the mean field description of balanced networks of binary neurons and apply the theory to clustered networks. We show that the stable fixed points of networks with clustered excitatory connectivity tend quickly towards firing rate saturation, which is generally inconsistent with experimental data. To remedy this shortcoming, we then present a novel perspective on networks with locally balanced clusters of both excitatory and inhibitory neuron populations. This approach allows for true multistability and moderate firing rates in activated clusters over a wide range of parameters. Our findings are supported by mean field theory and numerical network simulations. Finally, we discuss possible applications of the concept of joint excitatory and inhibitory clustering in future cortical network modelling studies.

[1]  Xiao-Jing Wang,et al.  Mean-Driven and Fluctuation-Driven Persistent Activity in Recurrent Networks , 2007, Neural Computation.

[2]  Prof. Dr. Valentino Braitenberg,et al.  Anatomy of the Cortex , 1991, Studies of Brain Function.

[3]  Nicolas Brunel,et al.  Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons , 2000, Journal of Computational Neuroscience.

[4]  Srdjan Ostojic,et al.  Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons , 2014, Nature Neuroscience.

[5]  E. Callaway,et al.  Fine-scale specificity of cortical networks depends on inhibitory cell type and connectivity , 2005, Nature Neuroscience.

[6]  Kevan A. C. Martin,et al.  Cortical Architecture , 2005, BVAI.

[7]  Haim Sompolinsky,et al.  Chaotic Balanced State in a Model of Cortical Circuits , 1998, Neural Computation.

[8]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[9]  A. Litwin-Kumar,et al.  Slow dynamics and high variability in balanced cortical networks with clustered connections , 2012, Nature Neuroscience.

[10]  Randolf Menzel,et al.  Cellular Adaptation Facilitates Sparse and Reliable Coding in Sensory Pathways , 2013, PLoS Comput. Biol..

[11]  H. Markram,et al.  Differential signaling via the same axon of neocortical pyramidal neurons. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[12]  A. Grinvald,et al.  Spontaneously emerging cortical representations of visual attributes , 2003, Nature.

[13]  Victor A. F. Lamme,et al.  Source (or Part of the following Source): Type Article Title Internal State of Monkey Primary Visual Cortex (v1) Predicts Figure Ground Perception Author(s) Internal State of Monkey Primary Visual Cortex (v1) Predicts Figure–ground Perception Materials and Methods , 2022 .

[14]  P. Dayan,et al.  Supporting Online Material Materials and Methods Som Text Figs. S1 to S9 References the Asynchronous State in Cortical Circuits , 2022 .

[15]  A. Grinvald,et al.  Dynamics of Ongoing Activity: Explanation of the Large Variability in Evoked Cortical Responses , 1996, Science.

[16]  Markus Diesmann,et al.  A multi-scale layer-resolved spiking network model of resting-state dynamics in macaque visual cortical areas , 2018, PLoS Comput. Biol..

[17]  Brent Doiron,et al.  Self-Organization of Microcircuits in Networks of Spiking Neurons with Plastic Synapses , 2015, PLoS Comput. Biol..

[18]  Andreas Kleinschmidt,et al.  Spontaneous local variations in ongoing neural activity bias perceptual decisions , 2008, Proceedings of the National Academy of Sciences.

[19]  Yi Wang,et al.  A Phase-Locking Analysis of Neuronal Firing Rhythms with Transcranial Magneto-Acoustical Stimulation Based on the Hodgkin-Huxley Neuron Model , 2017, Front. Comput. Neurosci..

[20]  Gordon Pipa,et al.  SORN: A Self-Organizing Recurrent Neural Network , 2009, Front. Comput. Neurosci..

[21]  Farzad Farkhooi,et al.  Adaptation reduces variability of the neuronal population code. , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[22]  Stefan Rotter,et al.  Dynamics of Competition between Subnetworks of Spiking Neuronal Networks in the Balanced State , 2015, PloS one.

[23]  T. Sejnowski,et al.  Reliability of spike timing in neocortical neurons. , 1995, Science.

[24]  T. M. Mayhew,et al.  Anatomy of the Cortex: Statistics and Geometry. , 1991 .

[25]  Brent Doiron,et al.  Balanced neural architecture and the idling brain , 2014, Front. Comput. Neurosci..

[26]  Olaf Sporns,et al.  The small world of the cerebral cortex , 2007, Neuroinformatics.

[27]  H. Markram,et al.  Interneurons of the neocortical inhibitory system , 2004, Nature Reviews Neuroscience.

[28]  Marc-Oliver Gewaltig,et al.  Dynamics of self-sustained asynchronous-irregular activity in random networks of spiking neurons with strong synapses , 2014, Front. Comput. Neurosci..

[29]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[30]  Xiao-Jing Wang,et al.  Mean-Field Theory of Irregularly Spiking Neuronal Populations and Working Memory in Recurrent Cortical Networks , 2003 .

[31]  Wulfram Gerstner,et al.  Cortical Dynamics in Presence of Assemblies of Densely Connected Weight-Hub Neurons , 2017, Front. Comput. Neurosci..

[32]  David J. Heeger,et al.  Neural variability: friend or foe? , 2015, Trends in Cognitive Sciences.

[33]  G. Shepherd,et al.  The neocortical circuit: themes and variations , 2015, Nature Neuroscience.

[34]  Everton J. Agnes,et al.  Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks , 2015, Nature Communications.

[35]  József Fiser,et al.  Spontaneous Cortical Activity Reveals Hallmarks of an Optimal Internal Model of the Environment , 2011, Science.

[36]  Sen Song,et al.  Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits , 2005, PLoS biology.

[37]  Stefan Rotter,et al.  Measurement of variability dynamics in cortical spike trains , 2008, Journal of Neuroscience Methods.

[38]  G. La Camera,et al.  Dynamics of Multistable States during Ongoing and Evoked Cortical Activity , 2015, The Journal of Neuroscience.

[39]  D. Amit,et al.  Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. , 1997, Cerebral cortex.

[40]  Steve I Perlmutter,et al.  Firing Properties of Spinal Interneurons during Voluntary Movement. I. State-Dependent Regularity of Firing , 2003, The Journal of Neuroscience.

[41]  A. Litwin-Kumar,et al.  Formation and maintenance of neuronal assemblies through synaptic plasticity , 2014, Nature Communications.

[42]  E. Callaway,et al.  Laminar sources of synaptic input to cortical inhibitory interneurons and pyramidal neurons , 2000, Nature Neuroscience.

[43]  A. Aertsen,et al.  Dynamic Encoding of Movement Direction in Motor Cortical Neurons , 2009, The Journal of Neuroscience.

[44]  Martin P. Nawrot,et al.  Analysis and Interpretation of Interval and Count Variability in Neural Spike Trains , 2010 .

[45]  R. Stein Some models of neuronal variability. , 1967, Biophysical journal.

[46]  A. Destexhe,et al.  The high-conductance state of neocortical neurons in vivo , 2003, Nature Reviews Neuroscience.

[47]  M. A. Smith,et al.  The spatial structure of correlated neuronal variability , 2016, Nature Neuroscience.

[48]  Timothée Masquelier,et al.  Neural variability, or lack thereof , 2013, Front. Comput. Neurosci..

[49]  W. Newsome,et al.  The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding , 1998, The Journal of Neuroscience.

[50]  Wulfram Gerstner,et al.  Neuronal Dynamics: From Single Neurons To Networks And Models Of Cognition , 2014 .

[51]  A. Larkman,et al.  Dendritic morphology of pyramidal neurones of the visual cortex of the rat: III. Spine distributions , 1991, The Journal of comparative neurology.

[52]  Moshe Abeles,et al.  The Doubly Balanced Network of Spiking Neurons: A Memory Model with High Capacity , 2003, NIPS.

[53]  K. Harris,et al.  Spontaneous Events Outline the Realm of Possible Sensory Responses in Neocortical Populations , 2009, Neuron.

[54]  Ad Aertsen,et al.  Frontiers in Neural Circuits Neural Circuits Precisely Timed Signal Transmission in Neocortical Networks with Reliable Intermediate-range Projections , 2009 .

[55]  A. Reyes,et al.  Spatial Profile of Excitatory and Inhibitory Synaptic Connectivity in Mouse Primary Auditory Cortex , 2012, The Journal of Neuroscience.

[56]  R. Reid,et al.  Low Response Variability in Simultaneously Recorded Retinal, Thalamic, and Cortical Neurons , 2000, Neuron.

[57]  B. Zemelman,et al.  The columnar and laminar organization of inhibitory connections to neocortical excitatory cells , 2010, Nature Neuroscience.

[58]  H. Sompolinsky,et al.  Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity , 1996, Science.

[59]  Gustavo Deco,et al.  Neural Network Mechanisms Underlying Stimulus Driven Variability Reduction , 2012, PLoS Comput. Biol..

[60]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[61]  Mauricio Barahona,et al.  Emergence of Slow-Switching Assemblies in Structured Neuronal Networks , 2015, PLoS Comput. Biol..

[62]  Peter beim Graben,et al.  Attractor and saddle node dynamics in heterogeneous neural fields , 2014 .

[63]  Wulfram Gerstner,et al.  Synaptic Plasticity in Neural Networks Needs Homeostasis with a Fast Rate Detector , 2013, PLoS Comput. Biol..

[64]  D. Amit,et al.  Effective neural response function for collective population states. , 1999, Network.

[65]  Andrew M. Clark,et al.  Stimulus onset quenches neural variability: a widespread cortical phenomenon , 2010, Nature Neuroscience.

[66]  Y. Kawaguchi,et al.  Cortical Inhibitory Cell Types Differentially Form Intralaminar and Interlaminar Subnetworks withExcitatory Neurons , 2009, The Journal of Neuroscience.

[67]  Wulfram Gerstner,et al.  Towards a theory of cortical columns: From spiking neurons to interacting neural populations of finite size , 2016, PLoS Comput. Biol..

[68]  T. Harkany,et al.  Pyramidal cell communication within local networks in layer 2/3 of rat neocortex , 2003, The Journal of physiology.

[69]  Raymond J. Dolan,et al.  Increased decision thresholds enhance information gathering performance in juvenile Obsessive-Compulsive Disorder (OCD) , 2017, PLoS Comput. Biol..

[70]  L. Pinneo On noise in the nervous system. , 1966, Psychological review.

[71]  P. Somogyi,et al.  Target-cell-specific facilitation and depression in neocortical circuits , 1998, Nature Neuroscience.

[72]  Ad Aertsen,et al.  Physiology and Impact of Horizontal Connections in Rat Neocortex. , 2015, Cerebral cortex.

[73]  A. Aertsen,et al.  Beyond the Cortical Column: Abundance and Physiology of Horizontal Connections Imply a Strong Role for Inputs from the Surround , 2011, Front. Neurosci..

[74]  Christian K. Machens,et al.  Variability in neural activity and behavior , 2014, Current Opinion in Neurobiology.