Neural Sampling by Irregular Gating Inhibition of Spiking Neurons and Attractor Networks

A long tradition in theoretical neuroscience casts sensory processing in the brain as the process of inferring the maximally consistent interpretations of imperfect sensory input. Recently it has been shown that Gamma-band inhibition can enable neural attractor networks to approximately carry out such a sampling mechanism. In this paper we propose a novel neural network model based on irregular gating inhibition, show analytically how it implements a Monte-Carlo Markov Chain (MCMC) sampler, and describe how it can be used to model networks of both neural attractors as well as of single spiking neurons. Finally we show how this model applied to spiking neurons gives rise to a new putative mechanism that could be used to implement stochastic synaptic weights in biological neural networks and in neuromorphic hardware.

[1]  Johannes Schemmel,et al.  Stochastic inference with deterministic spiking neurons , 2013, ArXiv.

[2]  József Fiser,et al.  Spontaneous Cortical Activity Reveals Hallmarks of an Optimal Internal Model of the Environment , 2011, Science.

[3]  Wolfgang Maass,et al.  Neural Dynamics as Sampling: A Model for Stochastic Computation in Recurrent Networks of Spiking Neurons , 2011, PLoS Comput. Biol..

[4]  Siddharth Joshi,et al.  Unsupervised Learning in Synaptic Sampling Machines , 2015, ArXiv.

[5]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[6]  Giacomo Indiveri,et al.  Synthesizing cognition in neuromorphic electronic systems , 2013, Proceedings of the National Academy of Sciences.

[7]  P. Berkes,et al.  Statistically Optimal Perception and Learning: from Behavior to Neural Representations , 2022 .

[8]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[9]  Peter E. Latham,et al.  Synaptic sampling: A connection between PSP variability and uncertainty explains neurophysiological observations , 2015, 1505.04544.

[10]  Giacomo Indiveri,et al.  Stochastic Interpretation of Quasi-periodic Event-based Systems , 2015, ArXiv.

[11]  John V. Arthur,et al.  The thermodynamic temperature of a rhythmic spiking network , 2010, ArXiv.

[12]  Stefan Habenschuss,et al.  Stochastic Computations in Cortical Microcircuit Models , 2013, PLoS Comput. Biol..

[13]  Paul Smolensky,et al.  Information processing in dynamical systems: foundations of harmony theory , 1986 .

[14]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[15]  G. Buzsáki,et al.  Mechanisms of gamma oscillations. , 2012, Annual review of neuroscience.

[16]  C. Gray,et al.  Dynamics of striate cortical activity in the alert macaque: I. Incidence and stimulus-dependence of gamma-band neuronal oscillations. , 2000, Cerebral cortex.

[17]  P. Fries.  A mechanism for cognitive dynamics: neuronal communication through neuronal coherence , 2005, Trends in Cognitive Sciences.

[18]  Ali Khiat,et al.  Implementation of a spike-based perceptron learning rule using TiO2−x memristors , 2015, Front. Neurosci..

[19]  Giacomo Indiveri,et al.  Rhythmic Inhibition Allows Neural Networks to Search for Maximally Consistent States , 2015, Neural Computation.

[20]  Gerhard Bohm,et al.  Introduction to Statistics and Data Analysis for Physicists , 2017 .

[21]  T. Branco,et al.  The probability of neurotransmitter release: variability and feedback control at single synapses , 2009, Nature Reviews Neuroscience.

[22]  Giacomo Indiveri,et al.  An event-based architecture for solving constraint satisfaction problems , 2015, Nature Communications.

[23]  Giacomo Indiveri,et al.  A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses , 2015, Front. Neurosci..

[24]  Wolfgang Maass,et al.  STDP enables spiking neurons to detect hidden causes of their inputs , 2009, NIPS.