Bayesian Spiking Neurons II: Learning

In the companion letter in this issue (Bayesian Spiking Neurons I: Inference), we showed that the dynamics of spiking neurons can be interpreted as a form of Bayesian integration, accumulating evidence over time about events in the external world or the body. We proceed to develop a theory of Bayesian learning in spiking neural networks, where the neurons learn to recognize temporal dynamics of their synaptic inputs. Meanwhile, successive layers of neurons learn hierarchical causal models for the sensory input. The corresponding learning rule is local, spike-time dependent, and highly nonlinear. This approach provides a principled description of spiking and plasticity rules maximizing information transfer, while limiting the number of costly spikes, between successive layers of neurons.

[1]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[2]  T. Poggio.  A theory of how the brain might work. , 1990, Cold Spring Harbor symposia on quantitative biology.

[3]  David W. Nauen,et al.  Coactivation and timing-dependent integration of synaptic potentiation and depression , 2005, Nature Neuroscience.

[4]  Sophie Denève,et al.  Bayesian Spiking Neurons I: Inference , 2008, Neural Computation.

[5]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[6]  C. Frith,et al.  From drugs to deprivation: a Bayesian framework for understanding models of psychosis , 2009, Psychopharmacology.

[7]  Adrienne L. Fairhall,et al.  Efficiency and ambiguity in an adaptive neural code , 2001, Nature.

[8]  Y. Dan,et al.  Spike-timing-dependent synaptic modification induced by natural spike trains , 2002, Nature.

[9]  Yang Dan,et al.  Synaptic Learning Rules, Cortical Circuits, and Visual Function , 2005, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[10]  W. Singer,et al.  Different voltage-dependent thresholds for inducing long-term depression and long-term potentiation in slices of rat visual cortex , 1990, Nature.

[11]  Stierlin.  Organization of Behavior. A Neuropsychological Theory , 1953 .

[12]  T. Bliss,et al.  A synaptic model of memory: long-term potentiation in the hippocampus , 1993, Nature.

[13]  R Linsker,et al.  From basic network principles to neural architecture: emergence of orientation columns. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[14]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[15]  K. Maccorquodale.  Organization of Behavior : A Neuropsychological Theory , 1951 .

[16]  Terrence J. Sejnowski,et al.  A Non-linear Information Maximisation Algorithm that Performs Blind Separation , 1994, NIPS.

[17]  P. J. Sjöström,et al.  Rate, Timing, and Cooperativity Jointly Determine Cortical Synaptic Plasticity , 2001, Neuron.

[18]  Martin J. Wainwright,et al.  Visual adaptation as optimal information transmission , 1999, Vision Research.

[19]  Walter Senn,et al.  Spike-Based Synaptic Plasticity and the Emergence of Direction Selective Simple Cells: Mathematical Analysis , 2003, Journal of Computational Neuroscience.

[20]  Shimon Ullman,et al.  Cortical Circuitry Implementing Graphical Models , 2009, Neural Computation.

[21]  H Barlow,et al.  Redundancy reduction revisited , 2001, Network.

[22]  Michael I. Jordan.  Learning in Graphical Models , 1999, NATO ASI Series.

[23]  William T. Freeman,et al.  Correctness of Belief Propagation in Gaussian Graphical Models of Arbitrary Topology , 1999, Neural Computation.

[24]  Y. Dan,et al.  Contribution of individual spikes in burst-induced long-term synaptic modification. , 2006, Journal of neurophysiology.

[25]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[26]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[27]  Brendan J. Frey,et al.  Graphical Models for Machine Learning and Digital Communication , 1998 .

[28]  Herman P. Snippe,et al.  Parameter Extraction from Population Codes: A Critical Assessment , 1996, Neural Computation.

[29]  D. Johnston,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997 .

[30]  Gianluigi Mongillo,et al.  Online Learning with Hidden Markov Models , 2008, Neural Computation.

[31]  Krzysztof J. Cios,et al.  Advances in neural information processing systems 7 , 1997 .

[32]  H. Markram,et al.  Redistribution of synaptic efficacy between neocortical pyramidal neurons , 1996, Nature.

[33]  M. Bear,et al.  Homosynaptic long-term depression in area CA1 of hippocampus and effects of N-methyl-D-aspartate receptor blockade. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[34]  Van Nostrand,et al.  Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm , 1967 .