Mixed Signal Learning by Spike Correlation Propagation in Feedback Inhibitory Circuits

The brain can learn and detect mixed input signals masked by various types of noise, and spike-timing-dependent plasticity (STDP) is the candidate synaptic level mechanism. Because sensory inputs typically have spike correlation, and local circuits have dense feedback connections, input spikes cause the propagation of spike correlation in lateral circuits; however, it is largely unknown how this secondary correlation generated by lateral circuits influences learning processes through STDP, or whether it is beneficial to achieve efficient spike-based learning from uncertain stimuli. To explore the answers to these questions, we construct models of feedforward networks with lateral inhibitory circuits and study how propagated correlation influences STDP learning, and what kind of learning algorithm such circuits achieve. We derive analytical conditions at which neurons detect minor signals with STDP, and show that depending on the origin of the noise, different correlation timescales are useful for learning. In particular, we show that non-precise spike correlation is beneficial for learning in the presence of cross-talk noise. We also show that by considering excitatory and inhibitory STDP at lateral connections, the circuit can acquire a lateral structure optimal for signal detection. In addition, we demonstrate that the model performs blind source separation in a manner similar to the sequential sampling approximation of the Bayesian independent component analysis algorithm. Our results provide a basic understanding of STDP learning in feedback circuits by integrating analyses from both dynamical systems and information theory.

[1]  Jean-Pascal Pfister,et al.  STDP in Adaptive Neurons Gives Close-To-Optimal Information Transmission , 2010, Front. Comput. Neurosci..

[2]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[3]  Benjamin A. Dunn,et al.  Recurrent inhibitory circuitry as a mechanism for grid formation , 2013, Nature Neuroscience.

[4]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[5]  M. Poo,et al.  Coincident Pre- and Postsynaptic Activity Modifies GABAergic Synapses by Postsynaptic Changes in Cl− Transporter Activity , 2003, Neuron.

[6]  Karl Deisseroth,et al.  Activation of Specific Interneurons Improves V1 Feature Selectivity and Visual Perception , 2012, Nature.

[7]  Matthieu Gilson,et al.  Stability versus Neuronal Specialization for STDP: Long-Tail Weight Distributions Solve the Dilemma , 2011, PloS one.

[8]  Jozsef Csicsvari,et al.  Dynamic Reconfiguration of Hippocampal Interneuron Circuits during Spatial Learning , 2013, Neuron.

[9]  Christoph von der Malsburg,et al.  The Correlation Theory of Brain Function , 1994 .

[10]  D. Ferster,et al.  Synchronous Membrane Potential Fluctuations in Neurons of the Cat Visual Cortex , 1999, Neuron.

[11]  Tomoki Fukai,et al.  A Simple Neural Network Exhibiting Selective Activation of Neuronal Ensembles: From Winner-Take-All to Winners-Share-All , 1997, Neural Computation.

[12]  Y. Dan,et al.  Coding of visual information by precisely correlated spikes in the lateral geniculate nucleus , 1998, Nature Neuroscience.

[13]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[14]  Sen Song,et al.  Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits , 2005, PLoS biology.

[15]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[16]  Joel Zylberberg,et al.  Inhibitory Interneurons Decorrelate Excitatory Cells to Drive Sparse Code Formation in a Spiking Model of V1 , 2013, The Journal of Neuroscience.

[17]  R. Christopher deCharms,et al.  Primary cortical representation of sounds by the coordination of action-potential timing , 1996, Nature.

[18]  H. Adesnik,et al.  Lateral competition for cortical space by layer-specific horizontal circuits , 2010, Nature.

[19]  S. Thorpe,et al.  Spike Timing Dependent Plasticity Finds the Start of Repeating Patterns in Continuous Spike Trains , 2008, PloS one.

[20]  Eytan Domany,et al.  Models of Neural Networks I , 1991 .

[21]  Chris C. Rodgers,et al.  Neural Correlates of Task Switching in Prefrontal Cortex and Primary Auditory Cortex in a Novel Stimulus Selection Task for Rodents , 2014, Neuron.

[22]  S. J. Roberts,et al.  Independent Component Analysis: Source Assessment Separation, a Bayesian Approach , 1998 .

[23]  Siu Kang,et al.  Bidirectional plasticity in fast-spiking GABA circuits by visual experience , 2009, Nature.

[24]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[25]  Henning Sprekeler,et al.  Inhibitory synaptic plasticity: spike timing-dependence and putative network function , 2013, Front. Neural Circuits.

[26]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[27]  S. Amari Dynamics of pattern formation in lateral-inhibition type neural fields , 1977, Biological Cybernetics.

[28]  Wolfgang Maass,et al.  Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity , 2013, PLoS Comput. Biol..

[29]  Johannes Schemmel,et al.  Stochastic inference with deterministic spiking neurons , 2013, ArXiv.

[30]  Jean-Pascal Pfister,et al.  Optimality Model of Unsupervised Spike-Timing-Dependent Plasticity: Synaptic Memory and Weight Distribution , 2007, Neural Computation.

[31]  Hermann Riecke,et al.  Mechanisms of pattern decorrelation by recurrent neuronal circuits , 2010, Nature Neuroscience.

[32]  M. Woodin,et al.  Spike-Timing Dependent Plasticity in Inhibitory Circuits , 2010, Front. Syn. Neurosci..

[33]  Arianna Maffei,et al.  Inhibitory Plasticity Dictates the Sign of Plasticity at Excitatory Synapses , 2014, The Journal of Neuroscience.

[34]  Robert A. Legenstein,et al.  What Can a Neuron Learn with Spike-Timing-Dependent Plasticity? , 2005, Neural Computation.

[35]  Jochen Triesch,et al.  Independent Component Analysis in Spiking Neurons , 2010, PLoS Comput. Biol..

[36]  Wolfgang Maass,et al.  Neural Dynamics as Sampling: A Model for Stochastic Computation in Recurrent Networks of Spiking Neurons , 2011, PLoS Comput. Biol..

[37]  A. Aertsen,et al.  Spiking activity propagation in neuronal networks: reconciling different perspectives on neural coding , 2010, Nature Reviews Neuroscience.

[38]  H. Abarbanel,et al.  Spike-timing-dependent plasticity of inhibitory synapses in the entorhinal cortex. , 2006, Journal of neurophysiology.

[39]  E. Oja Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[40]  Timothée Masquelier,et al.  Competitive STDP-Based Spike Pattern Learning , 2009, Neural Computation.

[41]  Vikrant Kapoor,et al.  Activity-dependent gating of lateral inhibition in the mouse olfactory bulb , 2008, Nature Neuroscience.

[42]  M. Poo,et al.  Spike-Timing-Dependent Plasticity of Neocortical Excitatory Synapses on Inhibitory Interneurons Depends on Target Cell Type , 2007, The Journal of Neuroscience.

[43]  K. Tanaka,et al.  Cross-Correlation Analysis of Interneuronal Connectivity in cat visual cortex. , 1981, Journal of neurophysiology.

[44]  Z. Mainen,et al.  Early events in olfactory processing. , 2006, Annual review of neuroscience.

[45]  Christoph E Schreiner,et al.  Spectrotemporal Processing Differences between Auditory Cortical Fast-Spiking and Regular-Spiking Neurons , 2008, The Journal of Neuroscience.

[46]  V. Murthy,et al.  An olfactory cocktail party: figure-ground segregation of odorants in rodents , 2014, Nature Neuroscience.

[47]  W. Bair,et al.  Correlated Firing in Macaque Visual Area MT: Time Scales and Relationship to Behavior , 2001, The Journal of Neuroscience.

[48]  G. Buzsáki,et al.  The log-dynamic brain: how skewed distributions affect network operations , 2014, Nature Reviews Neuroscience.

[49]  Josh H. McDermott The cocktail party problem , 2009, Current Biology.

[50]  Peter Somogyi,et al.  Cell Type-Specific Long-Term Plasticity at Glutamatergic Synapses onto Hippocampal Interneurons Expressing either Parvalbumin or CB1 Cannabinoid Receptor , 2010, The Journal of Neuroscience.

[51]  J. Deniau,et al.  Asymmetric spike-timing dependent plasticity of striatal nitric oxide-synthase interneurons , 2009, Neuroscience.

[52]  D. Brie,et al.  Separation of Non-Negative Mixture of Non-Negative Sources Using a Bayesian Approach and MCMC Sampling , 2006, IEEE Transactions on Signal Processing.

[53]  Mark Girolami,et al.  Extraction of independent signal sources using a deflationary exploratory projection pursuit network , 1997 .

[54]  J. Leo van Hemmen,et al.  Combined Hebbian development of geniculocortical and lateral connectivity in a model of primary visual cortex , 2001, Biological Cybernetics.

[55]  R. Kempter,et al.  Hebbian learning and spiking neurons , 1999 .

[56]  Kevin H. Knuth A Bayesian approach to source separation , 1999 .

[57]  Hongkui Zeng,et al.  Differential tuning and population dynamics of excitatory and inhibitory neurons reflect differences in local intracortical connectivity , 2011, Nature Neuroscience.

[58]  Barak A. Pearlmutter,et al.  Sparse Representations for the Cocktail Party Problem , 2006, The Journal of Neuroscience.

[59]  T. Sejnowski,et al.  Reliability of spike timing in neocortical neurons. , 1995, Science.

[60]  Tomoki Fukai,et al.  Interplay between Short- and Long-Term Plasticity in Cell-Assembly Formation , 2014, PloS one.

[61]  Stefan Habenschuss,et al.  Emergence of Optimal Decoding of Population Codes Through STDP , 2013, Neural Computation.

[62]  Ch. von der Malsburg,et al.  A neural cocktail-party processor , 1986, Biological Cybernetics.

[63]  Rainer W. Friedrich,et al.  Olfactory pattern classification by discrete neuronal network states , 2010, Nature.

[64]  A. Pouget,et al.  Not Noisy, Just Wrong: The Role of Suboptimal Inference in Behavioral Variability , 2012, Neuron.

[65]  R. Reid,et al.  Precisely correlated firing in cells of the lateral geniculate nucleus , 1996, Nature.

[66]  Matthieu Gilson,et al.  Emergence of network structure due to spike-timing-dependent plasticity in recurrent neuronal networks IV , 2009, Biological Cybernetics.

[67]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[68]  Paris Smaragdis,et al.  Blind separation of convolved mixtures in the frequency domain , 1998, Neurocomputing.

[69]  Matthieu Gilson,et al.  Spectral Analysis of Input Spike Trains by Spike-Timing-Dependent Plasticity , 2012, PLoS Comput. Biol..

[70]  Schuster,et al.  Separation of a mixture of independent signals using time delayed correlations. , 1994, Physical review letters.

[71]  W. Gerstner,et al.  Connectivity reflects coding: a model of voltage-based STDP with homeostasis , 2010, Nature Neuroscience.

[72]  Mark C. W. van Rossum,et al.  Correlation based learning from spike timing dependent plasticity , 2001, Neurocomputing.

[73]  M. A. Smith,et al.  Stimulus Dependence of Neuronal Correlation in Primary Visual Cortex of the Macaque , 2005, The Journal of Neuroscience.

[74]  W. Kinzel Physics of Neural Networks , 1990 .

[75]  Wulfram Gerstner,et al.  A neuronal learning rule for sub-millisecond temporal coding , 1996, Nature.

[76]  C. Schreiner,et al.  Modular organization of frequency integration in primary auditory cortex. , 2000, Annual review of neuroscience.

[77]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[78]  E. C. Cmm,et al.  on the Recognition of Speech, with , 2008 .

[79]  Matthieu Gilson,et al.  Emergence of network structure due to spike-timing-dependent plasticity in recurrent neuronal networks. I. Input selectivity–strengthening correlated input pathways , 2009, Biological Cybernetics.

[80]  J. Leo van Hemmen,et al.  Spontaneously emerging direction selectivity maps in visual cortex through STDP , 2005, Biological Cybernetics.

[81]  Erkki Oja,et al.  Neural Networks, Principal Components, and Subspaces , 1989, Int. J. Neural Syst..

[82]  Haim Sompolinsky,et al.  Learning Input Correlations through Nonlinear Temporally Asymmetric Hebbian Plasticity , 2003, The Journal of Neuroscience.

[83]  Alfredo Kirkwood,et al.  Adrenergic Gating of Hebbian Spike-Timing-Dependent Plasticity in Cortical Interneurons , 2013, The Journal of Neuroscience.

[84]  Matthieu Gilson,et al.  Emergence of network structure due to spike-timing-dependent plasticity in recurrent neuronal networks. II. Input selectivity—symmetry breaking , 2009, Biological Cybernetics.