Networks that learn the precise timing of event sequences

Neuronal circuits can learn and replay firing patterns evoked by sequences of sensory stimuli. After training, a brief cue can trigger a spatiotemporal pattern of neural activity similar to that evoked by a learned stimulus sequence. Network models show that such sequence learning can occur through the shaping of feedforward excitatory connectivity via long term plasticity. Previous models describe how event order can be learned, but they typically do not explain how precise timing can be recalled. We propose a mechanism for learning both the order and precise timing of event sequences. In our recurrent network model, long term plasticity leads to the learning of the sequence, while short term facilitation enables temporally precise replay of events. Learned synaptic weights between populations determine the time necessary for one population to activate another. Long term plasticity adjusts these weights so that the trained event times are matched during playback. While we chose short term facilitation as a time-tracking process, we also demonstrate that other mechanisms, such as spike rate adaptation, can fulfill this role. We also analyze the impact of trial-to-trial variability, showing how observational errors as well as neuronal noise result in variability in learned event times. The dynamics of the playback process determines how stochasticity is inherited in learned sequence timings. Future experiments that characterize such variability can therefore shed light on the neural mechanisms of sequence learning.

[1]  M. Treisman Temporal discrimination and the indifference interval. Implications for a model of the "internal clock". , 1963, Psychological monographs.

[2]  L. Pinneo On noise in the nervous system. , 1966, Psychological review.

[3]  Shun-ichi Amari,et al.  Learning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold Elements , 1972, IEEE Transactions on Computers.

[4]  J. Cowan,et al.  Excitatory and inhibitory interactions in localized populations of model neurons. , 1972, Biophysical journal.

[5]  T. Bliss,et al.  Long‐lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path , 1973, The Journal of physiology.

[6]  J. Gibbon Scalar expectancy theory and Weber's law in animal timing. , 1977 .

[7]  Roman Bek,et al.  Discourse on one way in which a quantum-mechanics language on the classical logical base can be built up , 1978, Kybernetika.

[8]  E. Oja Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[9]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[10]  C. Gardiner Handbook of Stochastic Methods , 1983 .

[11]  D Kleinfeld,et al.  Sequential state generation by model neural networks. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Christopher Miall,et al.  The Storage of Time Intervals Using Oscillating Neurons , 1989, Neural Computation.

[13]  Michael A. Arbib,et al.  Complex temporal sequence learning based on short-term memory , 1990 .

[14]  S. Grossberg,et al.  A neural network model of adaptively timed reinforcement learning and hippocampal dynamics. , 1992, Brain research. Cognitive brain research.

[15]  M. Bear,et al.  Homosynaptic long-term depression in area CA1 of hippocampus and effects of N-methyl-D-aspartate receptor blockade. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[16]  R. Malenka,et al.  The influence of prior synaptic activity on the induction of long-term potentiation. , 1992, Science.

[17]  T. Bliss,et al.  A synaptic model of memory: long-term potentiation in the hippocampus , 1993, Nature.

[18]  D. Brooks,et al.  Motor sequence learning: a study with positron emission tomography , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[19]  KD Miller A model for the development of simple cell receptive fields and the ordered arrangement of orientation columns through activity-dependent competition between ON- and OFF-center inputs , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[20]  J. Clements Transmitter timecourse in the synaptic cleft: its role in central synaptic function , 1996, Trends in Neurosciences.

[21]  H. Markram,et al.  Redistribution of synaptic efficacy between neocortical pyramidal neurons , 1996, Nature.

[22]  K. I. Blum,et al.  Functional significance of long-term potentiation for sequence learning and prediction. , 1996, Cerebral cortex.

[23]  M. Häusser,et al.  Estimating the Time Course of the Excitatory Synaptic Conductance in Neocortical Pyramidal Cells Using a Novel Voltage Jump Method , 1997, The Journal of Neuroscience.

[24]  H. Markram,et al.  Differential signaling via the same axon of neocortical pyramidal neurons. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[25]  O. Hikosaka,et al.  Transition of Brain Activation from Frontal to Parietal Areas in Visuomotor Sequence Learning , 1998, The Journal of Neuroscience.

[26]  Henry Markram,et al.  Neural Networks with Dynamic Synapses , 1998, Neural Computation.

[27]  R. Kempter,et al.  Hebbian learning and spiking neurons , 1999 .

[28]  N. Burgess,et al.  Memory for serial order: A network model of the phonological loop and its timing , 1999 .

[29]  N. Burgess Memory for Serial Order : A Network Model of the Phonological Loop and its Timing , 1999 .

[30]  G D Brown,et al.  Oscillator-based memory for serial order. , 2000, Psychological review.

[31]  D V Buonomano,et al.  Decoding Temporal Information: A Model Based on Short-Term Synaptic Plasticity , 2000, The Journal of Neuroscience.

[32]  Ruey-Beei Wu,et al.  Differential Signaling , 2000 .

[33]  E. Kandel The Molecular Biology of Memory Storage: A Dialogue Between Genes and Synapses , 2001, Science.

[34]  M. Bear,et al.  Visual Experience and Deprivation Bidirectionally Modify the Composition and Function of NMDA Receptors in Visual Cortex , 2001, Neuron.

[35]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[36]  Rajesh P. N. Rao,et al.  Spike-Timing-Dependent Hebbian Plasticity as Temporal Difference Learning , 2001, Neural Computation.

[37]  Morten H. Christiansen,et al.  Sequential learning in non-human primates , 2001, Trends in Cognitive Sciences.

[38]  P. J. Sjöström,et al.  Rate, Timing, and Cooperativity Jointly Determine Cortical Synaptic Plasticity , 2001, Neuron.

[39]  Kae Nakamura,et al.  Central mechanisms of motor skill learning , 2002, Current Opinion in Neurobiology.

[40]  W. Meck,et al.  Dissecting the Brain's Internal Clock: How Frontal–Striatal Circuitry Keeps Time and Shifts Attention , 2002, Brain and Cognition.

[41]  Xiao-Jing Wang,et al.  Probabilistic Decision Making by Slow Reverberation in Cortical Circuits , 2002, Neuron.

[42]  Wulfram Gerstner,et al.  Mathematical formulations of Hebbian learning , 2002, Biological Cybernetics.

[43]  Albert K. Lee,et al.  Memory of Sequential Experience in the Hippocampus during Slow Wave Sleep , 2002, Neuron.

[44]  Daniel Durstewitz,et al.  Self-Organizing Neural Integrator Predicts Interval Times through Climbing Activity , 2003, The Journal of Neuroscience.

[45]  Scott T. Grafton,et al.  Swinging in the brain: shared neural substrates for behaviors related to sequencing and music , 2003, Nature Neuroscience.

[46]  C. Palmer,et al.  Auditory feedback and memory for music performance: Sound evidence for an encoding effect , 2003, Memory & cognition.

[47]  J. Devin McAuley,et al.  Modeling effects of rhythmic context on perceived duration: a comparison of interval and entrainment approaches to short-interval timing. , 2003, Journal of experimental psychology. Human perception and performance.

[48]  Andreas V. M. Herz,et al.  A Universal Model for Spike-Frequency Adaptation , 2003, Neural Computation.

[49]  Peter Q Pfordresher,et al.  Auditory feedback in music performance: Evidence for a dissociation of sequencing and timing. , 2003, Journal of experimental psychology. Human perception and performance.

[50]  D. Hansel,et al.  How Spike Generation Mechanisms Determine the Neuronal Response to Fluctuating Inputs , 2003, The Journal of Neuroscience.

[51]  Haim Sompolinsky,et al.  Learning Input Correlations through Nonlinear Temporally Asymmetric Hebbian Plasticity , 2003, The Journal of Neuroscience.

[52]  Yuji Ikegaya,et al.  Synfire Chains and Cortical Songs: Temporal Modules of Cortical Activity , 2004, Science.

[53]  M. Bear,et al.  LTP and LTD An Embarrassment of Riches , 2004, Neuron.

[54]  W. Senn,et al.  Climbing Neuronal Activity as an Event-Based Cortical Representation of Time , 2004, The Journal of Neuroscience.

[55]  D. Tank,et al.  Persistent neural activity: prevalence and mechanisms , 2004, Current Opinion in Neurobiology.

[56]  Carson C. Chow,et al.  A Spiking Neuron Model for Binocular Rivalry , 2004, Journal of Computational Neuroscience.

[57]  I. Nelken,et al.  Multiple Time Scales of Adaptation in Auditory Cortex Neurons , 2004, The Journal of Neuroscience.

[58]  W. Meck,et al.  Neuroimaging of interval timing. , 2004, Brain research. Cognitive brain research.

[59]  C. Malsburg Self-organization of orientation sensitive cells in the striate cortex , 2004, Kybernetik.

[60]  Sen Song,et al.  Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits , 2005, PLoS biology.

[61]  Catalin V. Buhusi,et al.  What makes us tick? Functional and neural mechanisms of interval timing , 2005, Nature Reviews Neuroscience.

[62]  J. Doyon,et al.  Reorganization and plasticity in the adult brain during learning of motor skills , 2005, Current Opinion in Neurobiology.

[63]  Florentin Wörgötter,et al.  Temporal Sequence Learning, Prediction, and Control: A Review of Different Models and Their Relation to Biological Mechanisms , 2005, Neural Computation.

[64]  M Concetta Morrone,et al.  Saccadic eye movements cause compression of time as well as space , 2005, Nature Neuroscience.

[65]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[66]  W. Gerstner,et al.  Triplets of Spikes in a Model of Spike Timing-Dependent Plasticity , 2006, The Journal of Neuroscience.

[67]  Mark F Bear,et al.  Reward timing in the primary visual cortex. , 2006, Science.

[68]  Eric T. Shea-Brown,et al.  A firing rate model of Parkinsonian deficits in interval timing , 2006, Brain Research.

[69]  T. Soderling,et al.  Extrasynaptic Membrane Trafficking Regulated by GluR1 Serine 845 Phosphorylation Primes AMPA Receptors for Long-term Potentiation* , 2006, Journal of Biological Chemistry.

[70]  Joseph K Jun,et al.  Development of Neural Circuitry for Precise Temporal Sequences through Spontaneous Activity, Axon Remodeling, and Synaptic Plasticity , 2007, PloS one.

[71]  R. Zatorre,et al.  When the brain plays music: auditory–motor interactions in music perception and production , 2007, Nature Reviews Neuroscience.

[72]  M. Hasselmo,et al.  Spatial Representations of Hippocampal CA1 Neurons Are Modulated by Behavioral Context in a Hippocampus-Dependent Memory Task , 2007, The Journal of Neuroscience.

[73]  U. Karmarkar,et al.  Timing in the Absence of Clocks: Encoding Time in Neural Network States , 2007, Neuron.

[74]  Qingbo Wang,et al.  Feedforward Excitation and Inhibition Evoke Dual Modes of Firing in the Cat's Visual Thalamus during Naturalistic Viewing , 2007, Neuron.

[75]  Chun-I Yeh,et al.  Temporal precision in the neural code and the timescales of natural vision , 2007, Nature.

[76]  W. Abraham Metaplasticity: tuning synapses and networks for plasticity , 2008, Nature Reviews Neuroscience.

[77]  Joachim Haß,et al.  A neurocomputational model for optimal temporal processing , 2008, Journal of Computational Neuroscience.

[78]  Akihiro Yagi,et al.  Reduction of stimulus visibility compresses apparent time intervals , 2008, Nature Neuroscience.

[79]  John E. Schlerf,et al.  Dedicated and intrinsic models of time perception , 2008, Trends in Cognitive Sciences.

[80]  T. Bonhoeffer,et al.  Experience leaves a lasting structural trace in cortical circuits , 2008, Nature.

[81]  C. Alberini,et al.  Transcription factors in long-term memory and synaptic plasticity. , 2009, Physiological reviews.

[82]  Oliver Hobert Development of neural circuitry , 2009 .

[83]  Yonatan Loewenstein,et al.  Learning reward timing in cortex through reward dependent expression of synaptic plasticity , 2009, Proceedings of the National Academy of Sciences.

[84]  W. Maass,et al.  State-dependent computations: spatiotemporal processing in cortical networks , 2009, Nature Reviews Neuroscience.

[85]  W. Gerstner,et al.  Connectivity reflects coding: a model of voltage-based STDP with homeostasis , 2010, Nature Neuroscience.

[86]  N. Kraus,et al.  Music training for the development of auditory skills , 2010, Nature Reviews Neuroscience.

[87]  Richard Hans Robert Hahnloser,et al.  Spike-Time-Dependent Plasticity and Heterosynaptic Competition Organize Networks to Produce Long Scale-Free Sequences of Neural Activity , 2010, Neuron.

[88]  P. J. Sjöström,et al.  Functional specificity of local synaptic connections in neocortical networks , 2011, Nature.

[89]  Jonathan D. Cohen,et al.  A Model of Interval Timing by Neural Integration , 2011, The Journal of Neuroscience.

[90]  G. Buzsáki,et al.  Cell Assembly Sequences Arising from Spike Threshold Adaptation Keep Track of Time in the Hippocampus , 2011, The Journal of Neuroscience.

[91]  Thomas K. Berger,et al.  A synaptic organizing principle for cortical neuronal groups , 2011, Proceedings of the National Academy of Sciences.

[92]  C. Olson,et al.  Statistical learning of visual transitions in monkey inferotemporal cortex , 2011, Proceedings of the National Academy of Sciences.

[93]  H. Seo,et al.  A reservoir of time constants for memory traces in cortical neurons , 2011, Nature Neuroscience.

[94]  J. Pfister,et al.  A triplet spike-timing–dependent plasticity model generalizes the Bienenstock–Cooper–Munro rule to higher-order spatiotemporal correlations , 2011, Proceedings of the National Academy of Sciences.

[95]  N. Brunel,et al.  Calcium-based plasticity model explains sensitivity of synaptic changes to spike pattern, rate, and dendritic location , 2012, Proceedings of the National Academy of Sciences.

[96]  Valentin Dragoi,et al.  Image sequence reactivation in awake V4 networks , 2012, Proceedings of the National Academy of Sciences.

[97]  Janneke F. M. Jehee,et al.  Less Is More: Expectation Sharpens Representations in the Primary Visual Cortex , 2012, Neuron.

[98]  A. Litwin-Kumar,et al.  Slow dynamics and high variability in balanced cortical networks with clustered connections , 2012, Nature Neuroscience.

[99]  Y. Dan,et al.  Activity Recall in Visual Cortical Ensemble , 2012, Nature Neuroscience.

[100]  Mark C. W. van Rossum,et al.  Probabilistic inference of short-term synaptic plasticity in neocortical microcircuits , 2013, Front. Comput. Neurosci..

[101]  M. Carandini,et al.  Inhibition dominates sensory responses in awake cortex , 2012, Nature.

[102]  W. Senn,et al.  Matching Recall and Storage in Sequence Learning with Spiking Neural Networks , 2013, The Journal of Neuroscience.

[103]  W. Gerstner,et al.  Temporal whitening by power-law adaptation in neocortical neurons , 2013, Nature Neuroscience.

[104]  G. Borasio The role of palliative care in patients with neurological diseases , 2013, Nature Reviews Neurology.

[105]  Y. Dan,et al.  Representation of interval timing by temporally scalable firing patterns in rat prefrontal cortex , 2013, Proceedings of the National Academy of Sciences.

[106]  Mark F. Bear,et al.  Learned spatiotemporal sequence recognition and prediction in primary visual cortex , 2014, Nature Neuroscience.

[107]  David Kappel,et al.  STDP Installs in Winner-Take-All Circuits an Online Approximation to Hidden Markov Model Learning , 2014, PLoS Comput. Biol..

[108]  D. Buonomano,et al.  Temporal Perceptual Learning , 2014 .

[109]  W. Wildman,et al.  Theoretical Neuroscience , 2014 .

[110]  Sadegh Nabavi,et al.  Engineering a memory with LTD and LTP , 2014, Nature.

[111]  Tomonori Takeuchi,et al.  The synaptic plasticity and memory hypothesis: encoding, storage and persistence , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[112]  Brian Nils Lundstrom Modeling multiple time scale firing rate adaptation in a neural network of local field potentials , 2014, Journal of Computational Neuroscience.

[113]  W. Gerstner,et al.  Optimal Control of Transient Dynamics in Balanced Networks Supports Generation of Complex Movements , 2014, Neuron.

[114]  Mark S. Goldman,et al.  Memory without Feedback in a Neural Network , 2009, Neuron.