Efficient Computation Based on Stochastic Spikes

The speed and reliability of mammalian perception indicate that cortical computations can rely on very few action potentials per involved neuron. Together with the stochasticity of single-spike events in cortex, this appears to imply that large populations of redundant neurons are needed for rapid computations with action potentials. Here we demonstrate that very fast and precise computations can be realized also in small networks of stochastically spiking neurons. We present a generative network model for which we derive biologically plausible algorithms that perform spike-by-spike updates of the neuron's internal states and adaptation of its synaptic weights from maximizing the likelihood of the observed spike patterns. Paradigmatic computational tasks demonstrate the online performance and learning efficiency of our framework. The potential relevance of our approach as a model for cortical computation is discussed.

[1]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[2]  Arnaud Delorme,et al.  Spike-based strategies for rapid processing , 2001, Neural Networks.

[3]  Yves Burnod,et al.  Bayesian inference in populations of cortical neurons: a model of motion integration and segmentation in area MT , 1999, Biological Cybernetics.

[4]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[5]  J. Cowan,et al.  Excitatory and inhibitory interactions in localized populations of model neurons. , 1972, Biophysical journal.

[6]  Rajesh P. N. Rao.  Bayesian Computation in Recurrent Neural Circuits , 2004, Neural Computation.

[7]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[8]  Aapo Hyvärinen,et al.  Interpreting Neural Response Variability as Monte Carlo Sampling of the Posterior , 2002, NIPS.

[9]  I. Ohzawa,et al.  The effects of contrast on visual orientation and spatial frequency discrimination: a comparison of single cells and behavior. , 1987, Journal of neurophysiology.

[10]  P. Dettmar.  [Using the Fick's shifting hypothesis to explain dyschromatopsia in a model computation for data processing of disordered colored vision in man]. , 1972, Kybernetik.

[11]  W. Newsome,et al.  The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding , 1998, The Journal of Neuroscience.

[12]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[13]  William H. Richardson,et al.  Bayesian-Based Iterative Method of Image Restoration , 1972 .

[14]  Stefano Panzeri,et al.  On Decoding the Responses of a Population of Neurons from Short Time Windows , 1999, Neural Computation.

[15]  Michael J. Berry,et al.  The structure and precision of retinal spike trains. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[16]  M. Carandini,et al.  Summation and division by neurons in primate visual cortex. , 1994, Science.

[17]  穂鷹 良介.  Non-Linear Programming の計算法について , 1963 .

[18]  David Mumford,et al.  Pattern Theory: the Mathematics of Perception , 2002, math/0212400.

[19]  Rufin van Rullen,et al.  Rate Coding Versus Temporal Order Coding: What the Retinal Ganglion Cells Tell the Visual Cortex , 2001, Neural Computation.

[20]  Simon J. Thorpe,et al.  Coding static natural images using spiking event times: do neurons Cooperate? , 2004, IEEE Transactions on Neural Networks.

[21]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[22]  Arnaud Delorme,et al.  Face identification using one spike per neuron: resistance to image degradations , 2001, Neural Networks.

[23]  Wulfram Gerstner,et al.  Spiking Neuron Models , 2002 .

[24]  Nicolas Brunel,et al.  Dynamics of the Firing Probability of Noisy Integrate-and-Fire Neurons , 2002, Neural Computation.

[25]  H. Lantéri,et al.  Penalized maximum likelihood image restoration with positivity constraints:multiplicative algorithms , 2002 .

[26]  Frances S. Chance,et al.  Gain Modulation from Background Synaptic Input , 2002, Neuron.

[27]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[28]  Andreas K. Kreiter,et al.  Rapid contour integration in macaque monkeys , 2005, Vision Research.

[29]  H Markram,et al.  Dynamics of population rate codes in ensembles of neocortical neurons. , 2004, Journal of neurophysiology.

[30]  Konrad Paul Kording,et al.  Bayesian integration in sensorimotor learning , 2004, Nature.

[31]  Wulfram Gerstner,et al.  Spiking Neuron Models: An Introduction , 2002 .

[32]  William B. Levy,et al.  Energy Efficient Neural Codes , 1996, Neural Computation.

[33]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[34]  D. Heeger.  Normalization of cell responses in cat striate cortex , 1992, Visual Neuroscience.

[35]  M. Bethge,et al.  Optimal neural rate coding leads to bimodal firing rate distributions. , 2003, Network.

[36]  M. Bethge,et al.  Second order phase transition in neural rate coding: binary encoding is optimal for rapid signal transmission. , 2003, Physical review letters.

[37]  A. Pouget,et al.  Efficient computation and cue integration with noisy population codes , 2001, Nature Neuroscience.

[38]  Michael H. Herzog,et al.  Effects of grouping in contextual modulation , 2002, Nature.

[39]  L. Lucy.  An iterative technique for the rectification of observed distributions , 1974 .

[40]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.