A Non-spiking Neuron Model With Dynamic Leak to Avoid Instability in Recurrent Networks

Recurrent circuitry components are distributed widely within the brain, including both excitatory and inhibitory synaptic connections. Recurrent neuronal networks have potential stability problems, perhaps a predisposition to epilepsy. More generally, instability risks making internal representations of information unreliable. To assess the inherent stability properties of such recurrent networks, we tested a linear summation, non-spiking neuron model with and without a “dynamic leak”, corresponding to the low-pass filtering of synaptic input current by the RC circuit of the biological membrane. We first show that the output of this neuron model, in either of its two forms, follows its input at a higher fidelity than a wide range of spiking neuron models across a range of input frequencies. Then we constructed fully connected recurrent networks with equal numbers of excitatory and inhibitory neurons and randomly distributed weights across all synapses. When the networks were driven by pseudorandom sensory inputs with varying frequency, the recurrent network activity tended to induce high frequency self-amplifying components, sometimes evident as distinct transients, which were not present in the input data. The addition of a dynamic leak based on known membrane properties consistently removed such spurious high frequency noise across all networks. Furthermore, we found that the neuron model with dynamic leak imparts a network stability that seamlessly scales with the size of the network, conduction delays, the input density of the sensory signal and a wide range of synaptic weight distributions. Our findings suggest that neuronal dynamic leak serves the beneficial function of protecting recurrent neuronal circuitry from the self-induction of spurious high frequency signals, thereby permitting the brain to utilize this architectural circuitry component regardless of network size or recurrency.

[1]  Rodney J. Douglas,et al.  Inhibition in cortical circuits , 2009, Current Biology.

[2]  A. Zador,et al.  Balanced inhibition underlies tuning and sharpens spike timing in auditory cortex , 2003, Nature.

[3]  Guy M McKhann,et al.  A model for focal seizure onset, propagation, evolution, and progression , 2020, eLife.

[4]  H. Markram,et al.  Frequency and Dendritic Distribution of Autapses Established by Layer 5 Pyramidal Neurons in the Developing Rat Neocortex: Comparison with Synaptic Innervation of Adjacent Neurons of the Same Class , 1996, The Journal of Neuroscience.

[5]  Kaushik Roy,et al.  Towards Understanding the Effect of Leak in Spiking Neural Networks , 2020, Neurocomputing.

[6]  Bryan M. Hooks,et al.  Organization of Cortical and Thalamic Input to Pyramidal Neurons in Mouse Motor Cortex , 2013, The Journal of Neuroscience.

[7]  R. Douglas,et al.  A Quantitative Map of the Circuit of Cat Primary Visual Cortex , 2004, The Journal of Neuroscience.

[8]  Henrik Jörntell,et al.  Spike generation estimated from stationary spike trains in a variety of neurons in vivo , 2014, Front. Cell. Neurosci..

[9]  M. Steriade Synchronized activities of coupled oscillators in the cerebral cortex and thalamus at different levels of vigilance. , 1997, Cerebral cortex.

[10]  Kostas Tsakalis,et al.  Homeostasis of Brain Dynamics in Epilepsy: A Feedback Control Systems Perspective of Seizures , 2009, Annals of Biomedical Engineering.

[11]  K. Martin,et al.  Translaminar circuits formed by the pyramidal cells in the superficial layers of cat visual cortex , 2017, Brain Structure and Function.

[12]  Song Zhu,et al.  Robustness analysis for connection weight matrices of global exponential stability of stochastic recurrent neural networks , 2013, Neural Networks.

[13]  Viktor K. Jirsa,et al.  Phase-lags in large scale brain synchronization: Methodological considerations and in-silico analysis , 2018, PLoS Comput. Biol..

[14]  Kohitij Kar,et al.  Fast recurrent processing via ventral prefrontal cortex is needed by the primate ventral stream for robust core visual object recognition , 2020, bioRxiv.

[15]  Sen Song,et al.  Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits , 2005, PLoS biology.

[16]  G. Allen,et al.  Cerebrocerebellar communication systems. , 1974, Physiological reviews.

[17]  Jakob Tougaard,et al.  Signal detection theory, detectability and stochastic resonance effects , 2002, Biological Cybernetics.

[18]  Nicolas Brunel,et al.  Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons , 2000, Journal of Computational Neuroscience.

[19]  Joshua I. Sanders,et al.  Cortical interneurons that specialize in disinhibitory control , 2013, Nature.

[20]  Henning Sprekeler,et al.  Inhibitory Plasticity Balances Excitation and Inhibition in Sensory Pathways and Memory Networks , 2011, Science.

[21]  H. Jörntell,et al.  Channel current fluctuations conclusively explain neuronal encoding of internal potential into spike trains. , 2021, Physical review. E.

[22]  P. Somogyi,et al.  Massive Autaptic Self-Innervation of GABAergic Neurons in Cat Visual Cortex , 1997, The Journal of Neuroscience.

[23]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[24]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[25]  Alberto Mazzoni,et al.  Intracellular Dynamics in Cuneate Nucleus Neurons Support Self-Stabilizing Learning of Generalizable Tactile Representations , 2018, Front. Cell. Neurosci..

[26]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[27]  Alex Graves,et al.  Supervised Sequence Labelling , 2012 .

[28]  J. Zhu,et al.  Recurrent inhibitory circuitry in the deep layers of the rabbit superior colliculus , 2000, The Journal of physiology.

[29]  Olli Yli-Harja,et al.  Stochastic Differential Equation Model for Cerebellar Granule Cell Excitability , 2008, PLoS Comput. Biol..

[30]  Benjamin Lindner,et al.  Low-Pass Filtering of Information in the Leaky Integrate-and-Fire Neuron Driven by White Noise , 2014 .

[31]  Tim P Vogels,et al.  Signal Propagation and Logic Gating in Networks of Integrate-and-Fire Neurons , 2005, The Journal of Neuroscience.

[32]  H. Swadlow Fast-spike interneurons and feedforward inhibition in awake sensory neocortex. , 2003, Cerebral cortex.

[33]  K. Gopalsamy,et al.  Delay induced periodicity in a neural netlet of excitation and inhibition , 1996 .

[34]  H. Jörntell,et al.  In Vivo Analysis of Inhibitory Synaptic Inputs and Rebounds in Deep Cerebellar Nuclear Neurons , 2011, PloS one.

[35]  Henrik Jörntell,et al.  Cerebellar physiology: links between microcircuitry properties and sensorimotor functions , 2017, The Journal of physiology.

[36]  Khashayar Pakdaman,et al.  NOISE-INDUCED COHERENT OSCILLATIONS IN RANDOMLY CONNECTED NEURAL NETWORKS , 1998 .

[37]  Henrik Jörntell,et al.  Receptive Field Plasticity Profoundly Alters the Cutaneous Parallel Fiber Synaptic Input to Cerebellar Interneurons In Vivo , 2003, The Journal of Neuroscience.

[38]  Nicolas Brunel,et al.  Encoding of Naturalistic Stimuli by Local Field Potential Spectra in Networks of Excitatory and Inhibitory Neurons , 2008, PLoS Comput. Biol..

[39]  Song-Hai Shi,et al.  Generation of diverse cortical inhibitory interneurons , 2018, Wiley interdisciplinary reviews. Developmental biology.

[40]  Viktor K Jirsa,et al.  Neural Population Modes Capture Biologically Realistic Large Scale Network Dynamics , 2011, Bulletin of mathematical biology.

[41]  Jun Wang,et al.  Robustness Analysis of Global Exponential Stability of Recurrent Neural Networks in the Presence of Time Delays and Random Disturbances , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[42]  Michael Okun,et al.  Instantaneous correlation of excitation and inhibition during ongoing and sensory-evoked activities , 2008, Nature Neuroscience.

[43]  M. Scanziani,et al.  How Inhibition Shapes Cortical Activity , 2011, Neuron.

[44]  Haim Sompolinsky,et al.  Balanced excitation and inhibition are required for high-capacity, noise-robust neuronal selectivity , 2017, Proceedings of the National Academy of Sciences.

[45]  M. Carandini,et al.  Orientation tuning of input conductance, excitation, and inhibition in cat primary visual cortex. , 2000, Journal of neurophysiology.

[46]  M. Volgushev,et al.  Unique features of action potential initiation in cortical neurons , 2006, Nature.

[47]  Zhong-Wei Zhang,et al.  Maturation of layer V pyramidal neurons in the rat prefrontal cortex: intrinsic properties and synaptic function. , 2004, Journal of neurophysiology.

[48]  H. Jörntell,et al.  Questioning the role of sparse coding in the brain , 2015, Trends in Neurosciences.

[49]  J. J. Couey,et al.  Lateral inhibition by Martinotti interneurons is facilitated by cholinergic inputs in human and mouse neocortex , 2018, Nature Communications.