Supervised learning in spiking neural networks with FORCE training

Populations of neurons display an extraordinary diversity in the behaviors they affect and display. Machine learning techniques have recently emerged that allow us to create networks of model neurons that display behaviors of similar complexity. Here we demonstrate the direct applicability of one such technique, the FORCE method, to spiking neural networks. We train these networks to mimic dynamical systems, classify inputs, and store discrete sequences that correspond to the notes of a song. Finally, we use FORCE training to create two biologically motivated model circuits. One is inspired by the zebra finch and successfully reproduces songbird singing. The second network is motivated by the hippocampus and is trained to store and replay a movie scene. FORCE trained networks reproduce behaviors comparable in complexity to their inspired circuits and yield information not easily obtainable with other techniques, such as behavioral responses to pharmacological manipulations and spike timing statistics.FORCE training is a . Here the authors implement FORCE training in models of spiking neuronal networks and demonstrate that these networks can be trained to exhibit different dynamic behaviours.

[1]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[2]  M. Fee,et al.  Singing-related activity of identified HVC neurons in the zebra finch. , 2007, Journal of neurophysiology.

[3]  J. Lisman,et al.  The Theta-Gamma Neural Code , 2013, Neuron.

[4]  Marc W Howard,et al.  Time Cells in Hippocampal Area CA3 , 2016, The Journal of Neuroscience.

[5]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[6]  W. Gerstner,et al.  Predicting non-linear dynamics: a stable local learning scheme for recurrent spiking neural networks. , 2017 .

[7]  Péter Érdi,et al.  Computational theories on the function of theta oscillations , 2005, Biological Cybernetics.

[8]  Alla Borisyuk,et al.  Fluctuation-driven rhythmogenesis in an excitatory neuronal network with slow adaptation , 2008, Journal of Computational Neuroscience.

[9]  F. Nottebohm,et al.  Central control of song in the canary, Serinus canarius , 1976, The Journal of comparative neurology.

[10]  Joseph J. Paton,et al.  A Scalable Population Code for Time in the Striatum , 2015, Current Biology.

[11]  Adrienne L Fairhall,et al.  Constructing Precisely Computing Networks with Biophysical Spiking Neurons , 2014, The Journal of Neuroscience.

[12]  Sophie Denève,et al.  Enforcing balance allows local supervised learning in spiking recurrent networks , 2015, NIPS.

[13]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[14]  Peter Ford Dominey,et al.  Reservoir Computing Properties of Neural Dynamics in Prefrontal Cortex , 2016, PLoS Comput. Biol..

[15]  G. Buzsáki,et al.  Theta Oscillations Provide Temporal Windows for Local Circuit Computation in the Entorhinal-Hippocampal Loop , 2009, Neuron.

[16]  Lawrence Sirovich,et al.  On the Simulation of Large Populations of Neurons , 2004, Journal of Computational Neuroscience.

[17]  G. Buzsáki Theta Oscillations in the Hippocampus , 2002, Neuron.

[18]  Andrew M. Clark,et al.  Stimulus onset quenches neural variability: a widespread cortical phenomenon , 2010, Nature Neuroscience.

[19]  Mark M. Churchland,et al.  Using Firing-Rate Dynamics to Train Recurrent Networks of Spiking Model Neurons , 2016, 1601.07620.

[20]  S. Romani,et al.  Theta sequences are essential for internally generated hippocampal firing fields , 2014, Nature Neuroscience.

[21]  Henry Markram,et al.  On the computational power of circuits of spiking neurons , 2004, J. Comput. Syst. Sci..

[22]  H. Eichenbaum,et al.  Medial Entorhinal Cortex Selectively Supports Temporal Coding by Hippocampal Neurons , 2017, Neuron.

[23]  F. Wolf,et al.  Dynamic Flux Tubes Form Reservoirs of Stability in Neuronal Circuits , 2012 .

[24]  Abbott,et al.  Asynchronous states in networks of pulse-coupled oscillators. , 1993, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[25]  S. Strogatz,et al.  Stability of incoherence in a population of coupled oscillators , 1991 .

[26]  Bard Ermentrout,et al.  When inhibition not excitation synchronizes neural firing , 1994, Journal of Computational Neuroscience.

[27]  L. F. Abbott,et al.  Generating Coherent Patterns of Activity from Chaotic Neural Networks , 2009, Neuron.

[28]  Trevor Bekolay,et al.  A Large-Scale Model of the Functioning Brain , 2012, Science.

[29]  L. Abbott Stability and competition in multi-spike models of spike-timing dependent plasticity , 2010 .

[30]  D. Olton,et al.  Bidirectional Modulation of Scopolamine-Induced Working Memory Impairments by Muscarinic Activation of the Medial Septal Area , 1995, Neurobiology of Learning and Memory.

[31]  Lawrence Sirovich,et al.  Dynamics of neural populations: Stability and synchrony , 2006, Network.

[32]  Srdjan Ostojic,et al.  Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons , 2014, Nature Neuroscience.

[33]  Asohan Amarasingham,et al.  Internally Generated Cell Assembly Sequences in the Rat Hippocampus , 2008, Science.

[34]  Alexander Rivkind,et al.  Local Dynamics in Trained Recurrent Neural Networks. , 2015, Physical review letters.

[35]  Henry Markram,et al.  A Model for Real-Time Computation in Generic Neural Microcircuits , 2002, NIPS.

[36]  W. Gerstner,et al.  Connectivity reflects coding: a model of voltage-based STDP with homeostasis , 2010, Nature Neuroscience.

[37]  S. Strogatz From Kuramoto to Crawford: exploring the onset of synchronization in populations of coupled oscillators , 2000 .

[38]  Hilbert J. Kappen,et al.  Learning Universal Computations with Spikes , 2015, PLoS Comput. Biol..

[39]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[40]  Chris Eliasmith,et al.  Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems , 2004, IEEE Transactions on Neural Networks.

[41]  Carlos Lourenço Dynamical reservoir properties as network effects , 2006, ESANN.

[42]  L. Abbott,et al.  Eigenvalue spectra of random matrices for neural networks. , 2006, Physical review letters.

[43]  H. Sompolinsky,et al.  Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity , 1996, Science.

[44]  Haim Sompolinsky,et al.  Chaotic Balanced State in a Model of Cortical Circuits , 1998, Neural Computation.

[45]  W. Gerstner,et al.  Triplets of Spikes in a Model of Spike Timing-Dependent Plasticity , 2006, The Journal of Neuroscience.

[46]  Henry Markram,et al.  Neural Networks with Dynamic Synapses , 1998, Neural Computation.

[47]  Richard Hans Robert Hahnloser,et al.  An ultra-sparse code underliesthe generation of neural sequences in a songbird , 2002, Nature.

[48]  Lawrence Sirovich,et al.  Dynamics of Neuronal Populations: The Equilibrium Solution , 2000, SIAM J. Appl. Math..

[49]  S. Barry Cooper,et al.  Computability In Context: Computation and Logic in the Real World , 2009 .

[50]  Stefan Schliebs,et al.  Are probabilistic spiking neural networks suitable for reservoir computing? , 2011, The 2011 International Joint Conference on Neural Networks.

[51]  Christian K. Machens,et al.  Predictive Coding of Dynamical Variables in Balanced Spiking Networks , 2013, PLoS Comput. Biol..

[52]  Grzegorz M. Wójcik,et al.  Liquid State Machine Built of Hodgkin-Huxley Neurons , 2004, Informatica.

[53]  W. Klimesch EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis , 1999, Brain Research Reviews.

[54]  H. Eichenbaum Time cells in the hippocampus: a new dimension for mapping memories , 2014, Nature Reviews Neuroscience.

[55]  D. R. Euston,et al.  Fast-Forward Playback of Recent Memory Sequences in Prefrontal Cortex During Sleep , 2007, Science.

[56]  A. Leonardo,et al.  Ensemble Coding of Vocal Control in Birdsong , 2005, The Journal of Neuroscience.

[57]  M M Merzenich,et al.  Temporal information transformed into a spatial code by a neural network with realistic properties , 1995, Science.

[58]  Sidarta Ribeiro,et al.  Behaviourally driven gene expression reveals song nuclei in hummingbird brain , 2000, Nature.

[59]  Nicholas J. Priebe,et al.  The Emergence of Contrast-Invariant Orientation Tuning in Simple Cells of Cat Visual Cortex , 2007, Neuron.

[60]  L. F. Abbott,et al.  Building functional networks of spiking model neurons , 2016, Nature Neuroscience.

[61]  Dezhe Z. Jin,et al.  Support for a synaptic chain model of neuronal sequence generation , 2010, Nature.

[62]  Bruce W. Knight,et al.  Dynamics of Encoding in a Population of Neurons , 1972, The Journal of general physiology.

[63]  Wilten Nicola,et al.  Network bursting using experimentally constrained single compartment CA3 hippocampal neuron models with adaptation , 2011, Journal of Computational Neuroscience.

[64]  Matthew T. Kaufman,et al.  A neural network that finds a naturalistic solution for the production of muscle activity , 2015, Nature Neuroscience.

[65]  Nuo Li,et al.  Robust neuronal dynamics in premotor cortex during motor planning , 2016, Nature.

[66]  R. Mooney Different Subthreshold Mechanisms Underlie Song Selectivity in Identified HVc Neurons of the Zebra Finch , 2000, The Journal of Neuroscience.

[67]  G. Buzsáki Theta rhythm of navigation: Link between path integration and landmark navigation, episodic and semantic memory , 2005, Hippocampus.

[68]  Christopher D. Harvey,et al.  Recurrent Network Models of Sequence Generation and Memory , 2016, Neuron.

[69]  Peter Ford Dominey Complex sensory-motor sequence learning based on recurrent state representation and reinforcement learning , 1995, Biological Cybernetics.

[70]  Dean V. Buonomano,et al.  Neural Coding: Time Contraction and Dilation in the Striatum , 2015, Current Biology.

[71]  David Hansel,et al.  Asynchronous Rate Chaos in Spiking Neuronal Circuits , 2015 .

[72]  Benjamin Schrauwen,et al.  An overview of reservoir computing: theory, applications and implementations , 2007, ESANN.

[73]  G. Buzsáki,et al.  Forward and reverse hippocampal place-cell sequences during ripples , 2007, Nature Neuroscience.

[74]  Henry Markram,et al.  Fading memory and kernel properties of generic cortical microcircuit models , 2004, Journal of Physiology-Paris.

[75]  J. Cowan,et al.  Excitatory and inhibitory interactions in localized populations of model neurons. , 1972, Biophysical journal.

[76]  J. Csicsvari,et al.  Replay and Time Compression of Recurring Spike Sequences in the Hippocampus , 1999, The Journal of Neuroscience.

[77]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[78]  Henning Sprekeler,et al.  Inhibitory Plasticity Balances Excitation and Inhibition in Sensory Pathways and Memory Networks , 2011, Science.

[79]  Possible roles for GABAergic inhibition in the vocal control system of the zebra finch , 2000, Neuroreport.

[80]  Antoine Adamantidis,et al.  Causal evidence for the role of REM sleep theta rhythm in contextual memory consolidation , 2016, Science.

[81]  H. Eichenbaum,et al.  Hippocampal “Time Cells” Bridge the Gap in Memory for Discontiguous Events , 2011, Neuron.

[82]  Dean V. Buonomano,et al.  ROBUST TIMING AND MOTOR PATTERNS BY TAMING CHAOS IN RECURRENT NEURAL NETWORKS , 2012, Nature Neuroscience.

[83]  M. Konishi,et al.  Birdsong: from behavior to neuron. , 1985, Annual review of neuroscience.

[84]  G. Buzsáki Hippocampal sharp wave‐ripple: A cognitive biomarker for episodic memory and planning , 2015, Hippocampus.

[85]  Vreeswijk,et al.  Partial synchronization in populations of pulse-coupled oscillators. , 1996, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[86]  Andres D. Grosmark,et al.  Diversity in neural firing dynamics supports both rigid and learned hippocampal sequences , 2016, Science.

[87]  Michael S Brainard,et al.  Auditory-induced neural dynamics in sensory-motor circuitry predict learned temporal and sequential statistics of birdsong , 2016, Proceedings of the National Academy of Sciences.

[88]  Benjamin Schrauwen,et al.  Reservoir Computing Trends , 2012, KI - Künstliche Intelligenz.

[89]  Wolfgang Maass,et al.  Liquid State Machines: Motivation, Theory, and Applications , 2010 .

[90]  Duane Q. Nykamp,et al.  A Population Density Approach That Facilitates Large-Scale Modeling of Neural Networks: Extension to Slow Inhibitory Synapses , 2001, Neural Computation.

[91]  M. London,et al.  Sensitivity to perturbations in vivo implies high noise and suggests rate coding in cortex , 2010, Nature.

[92]  Matthew T. Kaufman,et al.  Neural population dynamics during reaching , 2012, Nature.

[93]  Nicholas T. Carnevale,et al.  ModelDB: A Database to Support Computational Neuroscience , 2004, Journal of Computational Neuroscience.

[94]  Wulfram Gerstner,et al.  Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network , 2017, eLife.

[95]  Jean-Jacques Slotine,et al.  Learning arbitrary dynamics in efficient, balanced spiking networks using local plasticity rules , 2017, AAAI 2017.

[96]  Terrence J. Sejnowski,et al.  Gradient Descent for Spiking Neural Networks , 2017, NeurIPS.

[97]  Evgueniy V. Lubenov,et al.  Hippocampal theta oscillations are travelling waves , 2009, Nature.

[98]  P. Bearman,et al.  Correction for Graupner and Brunel, Calcium-based plasticity model explains sensitivity of synaptic changes to spike pattern, rate, and dendritic location , 2012, Proceedings of the National Academy of Sciences.

[99]  Răzvan V. Florian Supervised learning in spiking neural networks , 2012 .

[100]  M. C. Ozturk,et al.  Computing with transiently stable states , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..