Somato-dendritic Synaptic Plasticity and Error-backpropagation in Active Dendrites

In the last decade dendrites of cortical neurons have been shown to nonlinearly combine synaptic inputs by evoking local dendritic spikes. It has been suggested that these nonlinearities raise the computational power of a single neuron, making it comparable to a 2-layer network of point neurons. But how these nonlinearities can be incorporated into the synaptic plasticity to optimally support learning remains unclear. We present a theoretically derived synaptic plasticity rule for supervised and reinforcement learning that depends on the timing of the presynaptic, the dendritic and the postsynaptic spikes. For supervised learning, the rule can be seen as a biological version of the classical error-backpropagation algorithm applied to the dendritic case. When modulated by a delayed reward signal, the same plasticity is shown to maximize the expected reward in reinforcement learning for various coding scenarios. Our framework makes specific experimental predictions and highlights the unique advantage of active dendrites for implementing powerful synaptic plasticity rules that have access to downstream information via backpropagation of action potentials.

[1]  M. Häusser,et al.  Dendritic Discrimination of Temporal Input Sequences in Cortical Neurons , 2010, Science.

[2]  Boris S. Gutkin,et al.  Contribution of sublinear and supralinear dendritic integration to neuronal computations , 2015, Front. Cell. Neurosci..

[3]  Haim Sompolinsky,et al.  Learning Precisely Timed Spikes , 2014, Neuron.

[4]  Walter Senn,et al.  Gradient estimation in dendritic reinforcement learning , 2012, Journal of mathematical neuroscience.

[5]  Johannes J. Letzkus,et al.  Learning Rules for Spike Timing-Dependent Plasticity Depend on Dendritic Synapse Location , 2006, The Journal of Neuroscience.

[6]  N. Spruston,et al.  Dendritic spikes induce single-burst long-term potentiation , 2007, Proceedings of the National Academy of Sciences.

[7]  Bartlett W. Mel,et al.  Pyramidal Neuron as Two-Layer Neural Network , 2003, Neuron.

[8]  Walter Senn,et al.  A Gradient Learning Rule for the Tempotron , 2009, Neural Computation.

[9]  Johannes J. Letzkus,et al.  Dendritic mechanisms controlling spike-timing-dependent synaptic plasticity , 2007, Trends in Neurosciences.

[10]  K. Holthoff,et al.  Single‐shock LTD by local dendritic spikes in pyramidal neurons of mouse visual cortex , 2004, The Journal of physiology.

[11]  Bartlett W. Mel,et al.  Impact of Active Dendrites and Structural Plasticity on the Memory Capacity of Neural Tissue , 2001, Neuron.

[12]  W. Gan,et al.  Branch-specific dendritic Ca2+ spikes cause persistent synaptic plasticity , 2015, Nature.

[13]  Matthew E Larkum,et al.  Synaptic clustering by dendritic signalling mechanisms , 2008, Current Opinion in Neurobiology.

[14]  Marc Timme,et al.  Statistical physics of neural systems with non-additive dendritic coupling , 2015, 1507.03881.

[15]  A. Polsky,et al.  Properties of basal dendrites of layer 5 pyramidal neurons: a direct patch-clamp recording study , 2007, Nature Neuroscience.

[16]  Henning Sprekeler,et al.  Functional Requirements for Reward-Modulated Spike-Timing-Dependent Plasticity , 2010, The Journal of Neuroscience.

[17]  Walter Senn,et al.  Reinforcement learning in dendritic structures , 2011, BMC Neuroscience.

[18]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[19]  Wulfram Gerstner,et al.  A neuronal learning rule for sub-millisecond temporal coding , 1996, Nature.

[20]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[21]  D. Johnston,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997 .

[22]  A. Holtmaat,et al.  Sensory-evoked LTP driven by dendritic plateau potentials in vivo , 2014, Nature.

[23]  Y. Dan,et al.  Spike-timing-dependent synaptic plasticity depends on dendritic location , 2005, Nature.

[24]  Guy Major,et al.  NMDA and GABAB (KIR) Conductances: The “Perfect Couple” for Bistability , 2013, The Journal of Neuroscience.

[25]  Jackie Schiller,et al.  Spatiotemporally graded NMDA spike/plateau potentials in basal dendrites of neocortical pyramidal neurons. , 2008, Journal of neurophysiology.

[26]  Paul A. Rhodes,et al.  The Properties and Implications of NMDA Spikes in Neocortical Pyramidal Cells , 2006, The Journal of Neuroscience.

[27]  H. Sompolinsky,et al.  The tempotron: a neuron that learns spike timing–based decisions , 2006, Nature Neuroscience.

[28]  E. Izhikevich Solving the distal reward problem through linkage of STDP and dopamine signaling , 2007, BMC Neuroscience.

[29]  Nace L. Golding,et al.  Dendritic spikes as a mechanism for cooperative long-term potentiation , 2002, Nature.

[30]  Bartlett W. Mel,et al.  Mechanisms underlying subunit independence in pyramidal neuron dendrites , 2013, Proceedings of the National Academy of Sciences.

[31]  Bartlett W. Mel,et al.  Computational subunits in thin dendrites of pyramidal cells , 2004, Nature Neuroscience.

[32]  A. Polsky,et al.  Synaptic Integration in Tuft Dendrites of Layer 5 Pyramidal Neurons: A New Unifying Principle , 2009, Science.

[33]  Boris S. Gutkin,et al.  Passive Dendrites Enable Single Neurons to Compute Linearly Non-separable Functions , 2013, PLoS Comput. Biol..

[34]  W. Senn,et al.  Matching Recall and Storage in Sequence Learning with Spiking Neural Networks , 2013, The Journal of Neuroscience.

[35]  Nathalie L Rochefort,et al.  Dendritic organization of sensory input to cortical neurons in vivo , 2010, Nature.

[36]  J. Schiller,et al.  NMDA spikes in basal dendrites of cortical pyramidal neurons , 2000, Nature.

[37]  K. Holthoff,et al.  A problem with Hebb and local spikes , 2002, Trends in Neurosciences.

[38]  P. J. Sjöström,et al.  Rate, Timing, and Cooperativity Jointly Determine Cortical Synaptic Plasticity , 2001, Neuron.

[39]  Judit K. Makara,et al.  Compartmentalized dendritic plasticity and input feature storage in neurons , 2008, Nature.

[40]  Jean-Pascal Pfister,et al.  Optimal Spike-Timing-Dependent Plasticity for Precise Action Potential Firing in Supervised Learning , 2005, Neural Computation.

[41]  Wolfgang Maass,et al.  Branch-Specific Plasticity Enables Self-Organization of Nonlinear Computation in Single Neurons , 2011, The Journal of Neuroscience.

[42]  J. Schiller,et al.  Active properties of neocortical pyramidal neuron dendrites. , 2013, Annual review of neuroscience.

[43]  J. Midtgaard,et al.  Synaptic integration in a model of cerebellar granule cells. , 1994, Journal of neurophysiology.

[44]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[45]  Jackie Schiller,et al.  Nonlinear dendritic processing determines angular tuning of barrel cortex neurons in vivo , 2012, Nature.

[46]  Urit Gordon,et al.  Plasticity Compartments in Basal Dendrites of Neocortical Pyramidal Neurons , 2006, The Journal of Neuroscience.

[47]  Nace L. Golding,et al.  Dendritic Sodium Spikes Are Variable Triggers of Axonal Action Potentials in Hippocampal CA1 Pyramidal Neurons , 1998, Neuron.

[48]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[49]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[50]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[51]  P. J. Sjöström,et al.  A Cooperative Switch Determines the Sign of Synaptic Plasticity in Distal Dendrites of Neocortical Pyramidal Neurons , 2006, Neuron.