Learning spatiotemporal signals using a recurrent spiking network that discretizes time

Learning to produce spatiotemporal sequences is a common task the brain has to solve. The same neural substrate may be used by the brain to produce different sequential behaviours. The way the brain learns and encodes such tasks remains unknown as current computational models do not typically use realistic biologically-plausible learning. Here, we propose a model where a spiking recurrent network of excitatory and inhibitory biophysical neurons drives a read-out layer: the dynamics of the recurrent network is constrained to encode time while the read-out neurons encode space. Space is then linked with time through plastic synapses that follow common Hebbian learning rules. We demonstrate that the model is able to learn spatiotemporal dynamics on a timescale that is behaviourally relevant. Learned sequences are robustly replayed during a regime of spontaneous activity. Author summary The brain has the ability to learn flexible behaviours on a wide range of time scales. Previous studies have successfully build spiking network models that learn a variety of computational tasks. However, often the learning involved is not local. Here, we investigate a model using biological-plausible plasticity rules for a specific computational task: spatiotemporal sequence learning. The architecture separates time and space into two different parts and this allows learning to bind space to time. Importantly, the time component is encoded into a recurrent network which exhibits sequential dynamics on a behavioural time scale. This network is then used as an engine to drive spatial read-out neurons. We demonstrate that the model can learn complicated spatiotemporal spiking dynamics, such as the song of a bird, and replay the song robustly.

[1]  Alex H. Williams,et al.  Unsupervised discovery of temporal sequences in high-dimensional datasets, with applications to neuroscience , 2018, bioRxiv.

[2]  Xin Jin,et al.  Shaping action sequences in basal ganglia circuits , 2015, Current Opinion in Neurobiology.

[3]  Timothy O'Leary,et al.  Fundamental bounds on learning performance in neural circuits , 2018, Proceedings of the National Academy of Sciences.

[4]  Masud Husain,et al.  Neural mechanisms of attending to items in working memory , 2019, Neuroscience & Biobehavioral Reviews.

[5]  Jochen Triesch,et al.  Robust development of synfire chains from multiple plasticity mechanisms , 2014, Front. Comput. Neurosci..

[6]  Tatsuo S. Okubo,et al.  Growth and splitting of neural sequences in songbird vocal development , 2015, Nature.

[7]  Henning Sprekeler,et al.  Inhibitory Plasticity Balances Excitation and Inhibition in Sensory Pathways and Memory Networks , 2011, Science.

[8]  K. D. Punta,et al.  An ultra-sparse code underlies the generation of neural sequences in a songbird , 2002 .

[9]  G. Buzsáki,et al.  Cell Assembly Sequences Arising from Spike Threshold Adaptation Keep Track of Time in the Hippocampus , 2011, The Journal of Neuroscience.

[10]  L. F. Abbott,et al.  Generating Coherent Patterns of Activity from Chaotic Neural Networks , 2009, Neuron.

[11]  Youngjin Park,et al.  Distinct role of flexible and stable encodings in sequential working memory , 2020, Neural Networks.

[12]  Markus Diesmann,et al.  Spike-Timing-Dependent Plasticity in Balanced Random Networks , 2007, Neural Computation.

[13]  Youngjin Park,et al.  Symmetry of learning rate in synaptic plasticity modulates formation of flexible and stable memories , 2017, Scientific Reports.

[14]  D. Debanne,et al.  Heterogeneity of Synaptic Plasticity at Unitary CA3–CA1 and CA3–CA3 Connections in Rat Hippocampal Slice Cultures , 1999, The Journal of Neuroscience.

[15]  James M Murray,et al.  Learning multiple variable-speed sequences in striatum via cortical tutoring , 2017, eLife.

[16]  Everton J. Agnes,et al.  Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks , 2015, Nature Communications.

[17]  Wulfram Gerstner,et al.  Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network , 2017, eLife.

[18]  Megan R. Carey,et al.  Spatial and Temporal Locomotor Learning in Mouse Cerebellum , 2019, Neuron.

[19]  Christopher D. Harvey,et al.  Choice-specific sequences in parietal cortex during a virtual-navigation decision task , 2012, Nature.

[20]  Wilten Nicola,et al.  Supervised learning in spiking neural networks with FORCE training , 2016, Nature Communications.

[21]  Yuji Ikegaya,et al.  Synfire Chains and Cortical Songs: Temporal Modules of Cortical Activity , 2004, Science.

[22]  Se-Bum Paik,et al.  Distinct role of flexible and stable encoding in sequential working memory , 2019, bioRxiv.

[23]  Wolfgang Maass,et al.  Searching for principles of brain computation , 2016, Current Opinion in Behavioral Sciences.

[24]  Sergey L. Gratiy,et al.  Fully integrated silicon probes for high-density recording of neural activity , 2017, Nature.

[25]  K. Lashley The problem of serial order in behavior , 1951 .

[26]  L. F. Abbott,et al.  Building functional networks of spiking model neurons , 2016, Nature Neuroscience.

[27]  Christopher D. Harvey,et al.  Recurrent Network Models of Sequence Generation and Memory , 2016, Neuron.

[28]  Yazan N. Billeh,et al.  Feedforward architectures driven by inhibitory interactions , 2017, Journal of Computational Neuroscience.

[29]  Ad Aertsen,et al.  From space to time: Spatial inhomogeneities lead to the emergence of spatiotemporal sequences in spiking neuronal networks , 2019, PLoS Comput. Biol..

[30]  M. Helmstaedter,et al.  Large-volume en-bloc staining for electron microscopy-based connectomics , 2015, Nature Communications.

[31]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[32]  Henry Markram,et al.  Synapses as dynamic memory buffers , 2002, Neural Networks.

[33]  Anders Lansner,et al.  Spike-Based Bayesian-Hebbian Learning of Temporal Sequences , 2016, PLoS Comput. Biol..

[34]  C. Clopath,et al.  The emergence of functional microcircuits in visual cortex , 2013, Nature.

[35]  Asohan Amarasingham,et al.  Internally Generated Cell Assembly Sequences in the Rat Hippocampus , 2008, Science.

[36]  Wilten Nicola,et al.  A diversity of interneurons and Hebbian plasticity facilitate rapid compressible learning in the hippocampus , 2019, Nature Neuroscience.

[37]  Francesca Mastrogiuseppe,et al.  Linking Connectivity, Dynamics, and Computations in Low-Rank Recurrent Neural Networks , 2017, Neuron.

[38]  Christopher M. Glaze,et al.  Temporal Structure in Zebra Finch Song: Implications for Motor Coding , 2006, The Journal of Neuroscience.

[39]  Hao Li,et al.  Optogenetic Editing Reveals the Hierarchical Organization of Learned Action Sequences , 2018, Cell.

[40]  Guy M. McKhann,et al.  Synfire Chains and Cortical SongsTemporal Modules of Cortical Activity , 2004 .

[41]  P. J. Sjöström,et al.  Rate, Timing, and Cooperativity Jointly Determine Cortical Synaptic Plasticity , 2001, Neuron.

[42]  Wulfram Gerstner,et al.  Excitable neuronal assemblies with adaptation as a building block of brain circuits for velocity-controlled signal propagation , 2018, PLoS Comput. Biol..

[43]  J. Tanji Sequential organization of multiple movements: involvement of cortical motor areas. , 2001, Annual review of neuroscience.

[44]  Asohan Amarasingham,et al.  Hippocampus Internally Generated Cell Assembly Sequences in the Rat , 2011 .

[45]  Mauricio Barahona,et al.  Emergence of Slow-Switching Assemblies in Structured Neuronal Networks , 2015, PLoS Comput. Biol..

[46]  Ryohei Yasuda,et al.  Somatostatin-Expressing Interneurons Enable and Maintain Learning-Dependent Sequential Activation of Pyramidal Neurons , 2019, Neuron.

[47]  Daniel Bullock,et al.  Learning and production of movement sequences: behavioral, neurophysiological, and modeling perspectives. , 2004, Human movement science.

[48]  David S. Lorberbaum,et al.  Genetic evidence that Nkx2.2 acts primarily downstream of Neurog3 in pancreatic endocrine lineage development , 2017, eLife.

[49]  Lorenz Pammer,et al.  Reliable Sequential Activation of Neural Assemblies by Single Pyramidal Cells in a Three-Layered Cortex , 2019, Neuron.

[50]  Simon X. Chen,et al.  Emergence of reproducible spatiotemporal activity during motor learning , 2014, Nature.

[51]  Ann M Graybiel,et al.  Neural representation of time in cortico-basal ganglia circuits , 2009, Proceedings of the National Academy of Sciences.

[52]  Richard Hans Robert Hahnloser,et al.  Spike-Time-Dependent Plasticity and Heterosynaptic Competition Organize Networks to Produce Long Scale-Free Sequences of Neural Activity , 2010, Neuron.

[53]  W. Senn,et al.  Matching Recall and Storage in Sequence Learning with Spiking Neural Networks , 2013, The Journal of Neuroscience.

[54]  A. Leonardo,et al.  Ensemble Coding of Vocal Control in Birdsong , 2005, The Journal of Neuroscience.

[55]  O. Hikosaka,et al.  Chunking during human visuomotor sequence learning , 2003, Experimental Brain Research.

[56]  Nicolas Brunel,et al.  Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons , 2000, Journal of Computational Neuroscience.

[57]  T. Tao Outliers in the spectrum of iid matrices with bounded rank perturbations , 2010 .

[58]  Richard Kempter,et al.  Memory replay in balanced recurrent networks , 2016, bioRxiv.

[59]  W. Gerstner,et al.  Connectivity reflects coding: a model of voltage-based STDP with homeostasis , 2010, Nature Neuroscience.

[60]  H. Sompolinsky,et al.  The tempotron: a neuron that learns spike timing–based decisions , 2006, Nature Neuroscience.

[61]  Moshe Abeles,et al.  Corticonics: Neural Circuits of Cerebral Cortex , 1991 .

[62]  Dean V. Buonomano,et al.  Encoding Time in Feedforward Trajectories of a Recurrent Neural Network Model , 2018, Neural Computation.

[63]  José Carlos Príncipe,et al.  Special issue on echo state networks and liquid state machines , 2007, Neural Networks.

[64]  J. Gibbon Scalar expectancy theory and Weber's law in animal timing. , 1977 .

[65]  Peter A. Appleby,et al.  Triphasic spike-timing-dependent plasticity organizes networks to produce robust sequences of neural activity , 2012, Front. Comput. Neurosci..

[66]  A. Litwin-Kumar,et al.  Formation and maintenance of neuronal assemblies through synaptic plasticity , 2014, Nature Communications.

[67]  Dean V. Buonomano,et al.  ROBUST TIMING AND MOTOR PATTERNS BY TAMING CHAOS IN RECURRENT NEURAL NETWORKS , 2012, Nature Neuroscience.

[68]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. , 2005, Journal of neurophysiology.