Stochastic variational learning in recurrent spiking networks

The ability to learn and perform statistical inference with biologically plausible recurrent networks of spiking neurons is an important step toward understanding perception and reasoning. Here we derive and investigate a new learning rule for recurrent spiking networks with hidden neurons, combining principles from variational learning and reinforcement learning. Our network defines a generative model over spike train histories and the derived learning rule has the form of a local Spike Timing Dependent Plasticity rule modulated by global factors (neuromodulators) conveying information about “novelty” on a statistically rigorous ground. Simulations show that our model is able to learn both stationary and non-stationary patterns of spike trains. We also propose one experiment that could potentially be performed with animals in order to test the dynamics of the predicted novelty signal.

[1]  D. Haar,et al.  Statistical Physics , 1971, Nature.

[2]  K. Miller,et al.  Ocular dominance column development: analysis and simulation. , 1989, Science.

[3]  R. H. White,et al.  Competitive Hebbian learning , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[4]  W. Singer,et al.  Long-term depression of excitatory synaptic transmission and its relationship to long-term potentiation , 1993, Trends in Neurosciences.

[5]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[6]  Wulfram Gerstner,et al.  A neuronal learning rule for sub-millisecond temporal coding , 1996, Nature.

[7]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[8]  D. Johnston,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997 .

[9]  R. Kempter,et al.  Hebbian learning and spiking neurons , 1999 .

[10]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[11]  A. Artola,et al.  Synaptic Activity Modulates the Induction of Bidirectional Synaptic Changes in Adult Mouse Hippocampus , 2000, The Journal of Neuroscience.

[12]  P. Dayan.  Helmholtz Machines and Wake-Sleep Learning , 2000 .

[13]  Michael I. Jordan,et al.  Bayesian parameter estimation via variational methods , 2000, Stat. Comput..

[14]  Isaac Meilijson,et al.  Distributed synchrony in a cell assembly of spiking neurons , 2001, Neural Networks.

[15]  P. J. Sjöström,et al.  Rate, Timing, and Cooperativity Jointly Determine Cortical Synaptic Plasticity , 2001, Neuron.

[16]  G. Bi,et al.  Synaptic modification by correlated activity: Hebb's postulate revisited. , 2001, Annual review of neuroscience.

[17]  Alison L Gibbs,et al.  On Choosing and Bounding Probability Metrics , 2002, math/0209021.

[18]  Wulfram Gerstner,et al.  Spiking Neuron Models , 2002 .

[19]  Q. Gu,et al.  Neuromodulatory transmitter systems in the cortex and their role in cortical plasticity , 2002, Neuroscience.

[20]  Alex M. Andrew,et al.  Spiking Neuron Models: Single Neurons, Populations, Plasticity , 2003 .

[21]  G. Rainer,et al.  Cognitive neuroscience: Neural mechanisms for detecting and remembering novel events , 2003, Nature Reviews Neuroscience.

[22]  L. Paninski.  Maximum likelihood estimation of cascade point-process neural encoding models , 2004, Network.

[23]  G. Edelman,et al.  Spike-timing dynamics of neuronal groups. , 2004, Cerebral cortex.

[24]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[25]  Konrad Paul Kording,et al.  Bayesian integration in sensorimotor learning , 2004, Nature.

[26]  Rufin van Rullen,et al.  Neurons Tune to the Earliest Spikes Through STDP , 2005, Neural Computation.

[27]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[28]  Matthew J. Beal,et al.  Variational Bayesian learning of directed graphical models with hidden variables , 2006 .

[29]  Jean-Pascal Pfister,et al.  Optimal Spike-Timing-Dependent Plasticity for Precise Action Potential Firing in Supervised Learning , 2005, Neural Computation.

[30]  Rémi Munos,et al.  Geometric Variance Reduction in Markov Chains: Application to Value Function and Gradient Estimation , 2005, J. Mach. Learn. Res..

[31]  Wulfram Gerstner,et al.  Predicting spike timing of neocortical pyramidal neurons by simple threshold models , 2006, Journal of Computational Neuroscience.

[32]  Shalabh Bhatnagar,et al.  Incremental Natural Actor-Critic Algorithms , 2007, NIPS.

[33]  Patrice Simardy,et al.  Learning Long-Term Dependencies with , 2007 .

[34]  Markus Diesmann,et al.  Spike-Timing-Dependent Plasticity in Balanced Random Networks , 2007, Neural Computation.

[35]  E. Izhikevich.  Solving the distal reward problem through linkage of STDP and dopamine signaling , 2007, BMC Neuroscience.

[36]  K. Obermayer,et al.  Cortical reorganization consistent with spike timing–but not correlation-dependent plasticity , 2007, Nature Neuroscience.

[37]  Karl J. Friston,et al.  Free-energy and the brain , 2007, Synthese.

[38]  Geoffrey E. Hinton,et al.  The Recurrent Temporal Restricted Boltzmann Machine , 2008, NIPS.

[39]  Sophie Denève,et al.  Bayesian Spiking Neurons I: Inference , 2008, Neural Computation.

[40]  Katsunori Kitano,et al.  Interplay between a phase response curve and spike-timing-dependent plasticity leading to wireless clustering. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[41]  Wolfgang Maass,et al.  STDP enables spiking neurons to detect hidden causes of their inputs , 2009, NIPS.

[42]  Ana Calabrese,et al.  Ocular dominance column , 2009, Scholarpedia.

[43]  Geoffrey E. Hinton,et al.  Deep Boltzmann Machines , 2009, AISTATS.

[44]  W. Gerstner,et al.  Connectivity reflects coding: A model of voltage-based spike-timing-dependent-plasticity with homeostasis , 2009 .

[45]  Matthieu Gilson,et al.  Emergence of network structure due to spike-timing-dependent plasticity in recurrent neuronal networks IV , 2009, Biological Cybernetics.

[46]  Henning Sprekeler,et al.  Functional Requirements for Reward-Modulated Spike-Timing-Dependent Plasticity , 2010, The Journal of Neuroscience.

[47]  Jochen Triesch,et al.  Independent Component Analysis in Spiking Neurons , 2010, PLoS Comput. Biol..

[48]  W. Gerstner,et al.  Connectivity reflects coding: a model of voltage-based STDP with homeostasis , 2010, Nature Neuroscience.

[49]  Wulfram Gerstner,et al.  Variational Learning for Recurrent Spiking Networks , 2011, NIPS.

[50]  Jean-Pascal Pfister,et al.  Sequence learning with hidden units in spiking neural networks , 2011, NIPS.

[51]  Wolfgang Maass,et al.  Probabilistic Inference in General Graphical Models through Sampling in Stochastic Networks of Spiking Neurons , 2011, PLoS Comput. Biol..

[52]  József Fiser,et al.  Spontaneous Cortical Activity Reveals Hallmarks of an Optimal Internal Model of the Environment , 2011, Science.

[53]  Stefan Habenschuss,et al.  Homeostatic plasticity in Bayesian spiking networks as Expectation Maximization with posterior constraints , 2012, NIPS.

[54]  Louis Yuanlong Shao.  Linear-Nonlinear-Poisson Neurons Can Do Inference On Deep Boltzmann Machines , 2013, ICLR.

[55]  Wolfgang Maass,et al.  Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity , 2013, PLoS Comput. Biol..

[56]  W. Senn,et al.  Matching Recall and Storage in Sequence Learning with Spiking Neural Networks , 2013, The Journal of Neuroscience.

[57]  Zoubin Ghahramani.  Variational Bayesian Learning , 2000, Variational Bayesian Learning Theory.