Associative learning of classical conditioning as an emergent property of spatially extended spiking neural circuits with synaptic plasticity

Associative learning of temporally disparate events is of fundamental importance for perceptual and cognitive functions. Previous studies of the neural mechanisms of such association have been mainly focused on individual neurons or synapses, often with an assumption that there is persistent neural firing activity that decays slowly. However, experimental evidence supporting such firing activity for associative learning is still inconclusive. Here we present a novel, alternative account of associative learning in the context of classical conditioning, demonstrating that it is an emergent property of a spatially extended, spiking neural circuit with spike-timing dependent plasticity and short term synaptic depression. We show that both the conditioned and unconditioned stimuli can be represented by spike sequences which are produced by wave patterns propagating through the network, and that the interactions of these sequences are timing-dependent. After training, the occurrence of the sequence encoding the conditioned stimulus (CS) naturally regenerates that encoding the unconditioned stimulus (US), therefore resulting in association between them. Such associative learning based on interactions of spike sequences can happen even when the timescale of their separation is significantly larger than that of individual neurons. In particular, our network model is able to account for the temporal contiguity property of classical conditioning, as observed in behavioral studies. We further show that this emergent associative learning in our network model is quite robust to noise perturbations. Our results therefore demonstrate that associative learning of temporally disparate events can happen in a distributed way at the level of neural circuits.

[1]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[2]  Y. Dan,et al.  Spike timing-dependent plasticity: from synapse to perception. , 2006, Physiological reviews.

[3]  James K. Kroger,et al.  Cross-modal and cross-temporal association in neurons of frontal cortex , 2000, Nature.

[4]  W. Regehr,et al.  Short-term synaptic plasticity. , 2002, Annual review of physiology.

[5]  G. Edelman,et al.  Spike-timing dynamics of neuronal groups. , 2004, Cerebral cortex.

[6]  Wulfram Gerstner,et al.  A History of Spike-Timing-Dependent Plasticity , 2011, Front. Syn. Neurosci..

[7]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[8]  Y. Dan,et al.  Activity Recall in Visual Cortical Ensemble , 2012, Nature Neuroscience.

[9]  György Buzsáki,et al.  Neural Syntax: Cell Assemblies, Synapsembles, and Readers , 2010, Neuron.

[10]  G. Buzsáki,et al.  Local Generation and Propagation of Ripples along the Septotemporal Axis of the Hippocampus , 2013, The Journal of Neuroscience.

[11]  R. L. Beurle Properties of a mass of cells capable of regenerating pulses , 1956, Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences.

[12]  Robert A. Frazor,et al.  Standing Waves and Traveling Waves Distinguish Two Circuits in Visual Cortex , 2007, Neuron.

[13]  G. Buzsáki,et al.  Traveling Theta Waves along the Entire Septotemporal Axis of the Hippocampus , 2012, Neuron.

[14]  Michael Breakspear,et al.  A Dendritic Mechanism for Decoding Traveling Waves: Principles and Applications to Motor Cortex , 2013, PLoS Comput. Biol..

[15]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[16]  M. Häusser,et al.  Dendritic Discrimination of Temporal Input Sequences in Cortical Neurons , 2010, Science.

[17]  German Barrionuevo,et al.  Synaptic targets of the intrinsic axon collaterals of supragranular pyramidal neurons in monkey prefrontal cortex , 2001, The Journal of comparative neurology.

[18]  S. Rumpel,et al.  Discrete Neocortical Dynamics Predict Behavioral Categorization of Sounds , 2012, Neuron.

[19]  Asohan Amarasingham,et al.  Internally Generated Cell Assembly Sequences in the Rat Hippocampus , 2008, Science.

[20]  N. Hatsopoulos,et al.  Propagating waves mediate information transfer in the motor cortex , 2006, Nature Neuroscience.

[21]  Florentin Wörgötter,et al.  Temporal Sequence Learning, Prediction, and Control: A Review of Different Models and Their Relation to Biological Mechanisms , 2005, Neural Computation.

[22]  Li I. Zhang,et al.  A critical window for cooperation and competition among developing retinotectal synapses , 1998, Nature.

[23]  Ruey-Beei Wu,et al.  Differential Signaling , 2000 .

[24]  James C. Smith,et al.  A comparison of taste aversions induced by radiation and lithium chloride in CS-US and US-CS paradigms. , 1974, Journal of comparative and physiological psychology.

[25]  D. R. Euston,et al.  Fast-Forward Playback of Recent Memory Sequences in Prefrontal Cortex During Sleep , 2007, Science.

[26]  Christopher D. Harvey,et al.  Choice-specific sequences in parietal cortex during a virtual-navigation decision task , 2012, Nature.

[27]  L. Abbott,et al.  Extending the effects of spike-timing-dependent plasticity to behavioral timescales. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[28]  Jian-Young Wu,et al.  Propagating Waves of Activity in the Neocortex: What They Are, What They Do , 2008, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[29]  R. Rescorla Behavioral studies of Pavlovian conditioning. , 1988, Annual review of neuroscience.

[30]  J. Rawlins,et al.  The septo-hippocampal system and cognitive mapping , 1982, Behavioural Brain Research.

[31]  K. Takagaki,et al.  Interactions between two propagating waves in rat visual cortex , 2012, Neuroscience.

[32]  Moshe Abeles,et al.  Corticonics: Neural Circuits of Cerebral Cortex , 1991 .

[33]  Feng Qi Han,et al.  Reverberation of Recent Visual Experience in Spontaneous Cortical Waves , 2008, Neuron.

[34]  A. Aertsen,et al.  Spiking activity propagation in neuronal networks: reconciling different perspectives on neural coding , 2010, Nature Reviews Neuroscience.

[35]  Wulfram Gerstner,et al.  Spike-timing dependent plasticity , 2010, Scholarpedia.

[36]  G Horn,et al.  An analysis of spontaneous impulse activity of units in the striate cortex of unrestrained cats , 1966, The Journal of physiology.

[37]  L. Abbott,et al.  Limits on the memory storage capacity of bounded synapses , 2007, Nature Neuroscience.

[38]  Tatsuo K Sato,et al.  Traveling Waves in Visual Cortex , 2012, Neuron.

[39]  Mathew E. Diamond,et al.  Hippocampal Representation of Touch-Guided Behavior in Rats: Persistent and Independent Traces of Stimulus and Reward Location , 2011, PloS one.

[40]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[41]  M. Hasselmo,et al.  The hippocampus as an associator of discontiguous events , 1998, Trends in Neurosciences.

[42]  S. R. Coleman,et al.  Classical conditioning of the rabbit's nictitating membrane response at backward, simultaneous, and forward CS-US intervals. , 1969, Journal of comparative and physiological psychology.

[43]  H. Markram,et al.  Interneurons of the neocortical inhibitory system , 2004, Nature Reviews Neuroscience.

[44]  H. Eichenbaum,et al.  The Hippocampus, Memory, and Place Cells Is It Spatial Memory or a Memory Space? , 1999, Neuron.

[45]  Nestor A. Schmajuk,et al.  Classical conditioning , 2008, Scholarpedia.

[46]  Cees van Leeuwen,et al.  Distributed Dynamical Computation in Neural Circuits with Propagating Coherent Activity Patterns , 2009, PLoS Comput. Biol..

[47]  B L McNaughton,et al.  Path Integration and Cognitive Mapping in a Continuous Attractor Neural Network Model , 1997, The Journal of Neuroscience.

[48]  Daniel Lehmann,et al.  Modeling Compositionality by Dynamic Binding of Synfire Chains , 2004, Journal of Computational Neuroscience.

[49]  I. Pavlov Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex , 1929 .

[50]  B. Raman,et al.  Sparse odor representation and olfactory learning , 2008, Nature Neuroscience.

[51]  S. Laughlin,et al.  Energy limitation as a selective pressure on the evolution of sensory systems , 2008, Journal of Experimental Biology.

[52]  R. F. Thompson,et al.  Modeling the neural substrates of associative learning and memory: a computational approach. , 1987, Psychological review.

[53]  John H. C. Palmer,et al.  Formation and Regulation of Dynamic Patterns in Two-Dimensional Spiking Neural Circuits with Spike-Timing-Dependent Plasticity , 2013, Neural Computation.

[54]  M. Wilson,et al.  Coordinated memory replay in the visual cortex and hippocampus during sleep , 2007, Nature Neuroscience.

[55]  F. Haiss,et al.  Spatiotemporal Dynamics of Cortical Sensorimotor Integration in Behaving Mice , 2007, Neuron.

[56]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[57]  I. Pavlov,et al.  Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex , 2010, Annals of Neurosciences.

[58]  H. Markram,et al.  Differential signaling via the same axon of neocortical pyramidal neurons. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[59]  Evgueniy V. Lubenov,et al.  Hippocampal theta oscillations are travelling waves , 2009, Nature.

[60]  J. Fuster,et al.  Unit activity in monkey parietal cortex related to haptic perception and temporary memory , 2004, Experimental Brain Research.

[61]  S. Nelson,et al.  Short-Term Depression at Thalamocortical Synapses Contributes to Rapid Adaptation of Cortical Sensory Responses In Vivo , 2002, Neuron.