Probabilistic inference in discrete spaces can be implemented into networks of LIF neurons

The means by which cortical neural networks are able to efficiently solve inference problems remains an open question in computational neuroscience. Recently, abstract models of Bayesian computation in neural circuits have been proposed, but they lack a mechanistic interpretation at the single-cell level. In this article, we describe a complete theoretical framework for building networks of leaky integrate-and-fire neurons that can sample from arbitrary probability distributions over binary random variables. We test our framework for a model inference task based on a psychophysical phenomenon (the Knill-Kersten optical illusion) and further assess its performance when applied to randomly generated distributions. As the local computations performed by the network strongly depend on the interaction between neurons, we compare several types of couplings mediated by either single synapses or interneuron chains. Due to its robustness to substrate imperfections such as parameter noise and background noise correlations, our model is particularly interesting for implementation on novel, neuro-inspired computing architectures, which can thereby serve as a fast, low-power substrate for solving real-world inference problems.

[1]  Ad Aertsen,et al.  Stable propagation of synchronous spiking in cortical neural networks , 1999, Nature.

[2]  Wolfgang Maass,et al.  Probabilistic Inference in General Graphical Models through Sampling in Stochastic Networks of Spiking Neurons , 2011, PLoS Comput. Biol..

[3]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model , 2009, Scholarpedia.

[4]  Giacomo Indiveri,et al.  A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity , 2006, IEEE Transactions on Neural Networks.

[5]  Alain Destexhe,et al.  High-conductance state , 2007, Scholarpedia.

[6]  R. Rosenfeld Nature , 2009, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[7]  D. Knill,et al.  Apparent surface curvature affects lightness perception , 1991, Nature.

[8]  Nicholas T. Carnevale,et al.  The NEURON Simulation Environment , 1997, Neural Computation.

[9]  Ad Aertsen,et al.  Functional consequences of correlated excitatory and inhibitory conductances in cortical networks , 2010, Journal of Computational Neuroscience.

[10]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. , 2005, Journal of neurophysiology.

[11]  A. Destexhe,et al.  The high-conductance state of neocortical neurons in vivo , 2003, Nature Reviews Neuroscience.

[12]  Jim D. Garside,et al.  Overview of the SpiNNaker System Architecture , 2013, IEEE Transactions on Computers.

[13]  Rajesh P. N. Rao Hierarchical Bayesian Inference in Networks of Spiking Neurons , 2004, NIPS.

[14]  H. Markram,et al.  The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Michael N. Shadlen,et al.  Probabilistic reasoning by neurons , 2007, Nature.

[16]  J. Herskowitz,et al.  Proceedings of the National Academy of Sciences, USA , 1996, Current Biology.

[17]  M. V. Rossum,et al.  In Neural Computation , 2022 .

[18]  Peter Tino,et al.  IEEE Transactions on Neural Networks , 2009 .

[19]  David S. Touretzky,et al.  Advances in neural information processing systems 2 , 1989 .

[20]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[21]  Nicholas T. Carnevale,et al.  Neuron simulation environment , 2007, Scholarpedia.

[22]  T. Sejnowski,et al.  Reliability of spike timing in neocortical neurons. , 1995, Science.

[23]  Wolfgang Maass,et al.  Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity , 2013, PLoS Comput. Biol..

[24]  Johannes Schemmel,et al.  Stochastic inference with deterministic spiking neurons , 2013, ArXiv.

[25]  József Fiser,et al.  Spontaneous Cortical Activity Reveals Hallmarks of an Optimal Internal Model of the Environment , 2011, Science.

[26]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[27]  Wolfgang Maass,et al.  Neural Dynamics as Sampling: A Model for Stochastic Computation in Recurrent Networks of Spiking Neurons , 2011, PLoS Comput. Biol..

[28]  Wolfgang Maass,et al.  Belief Propagation in Networks of Spiking Neurons , 2009, Neural Computation.

[29]  Fabio Gagliardi Cozman,et al.  Random Generation of Bayesian Networks , 2002, SBIA.

[30]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[31]  Giacomo Indiveri,et al.  Real-Time Classification of Complex Patterns Using Spike-Based Learning in Neuromorphic VLSI , 2009, IEEE Transactions on Biomedical Circuits and Systems.

[32]  David Kappel,et al.  STDP Installs in Winner-Take-All Circuits an Online Approximation to Hidden Markov Model Learning , 2014, PLoS Comput. Biol..

[33]  Marc-Oliver Gewaltig,et al.  NEST: An Environment for Neural Systems Simulations , 2003 .

[34]  C. W. Wu,et al.  Proceedings of the 2007 IEEE International Symposium on Circuits and Systems (ISCAS) , 2007 .

[35]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[36]  Sophie Denève,et al.  Bayesian Spiking Neurons I: Inference , 2008, Neural Computation.

[37]  Pierre Yger,et al.  PyNN: A Common Interface for Neuronal Network Simulators , 2008, Front. Neuroinform..

[38]  Robert A. Legenstein,et al.  Ensembles of Spiking Neurons with Noise Support Optimal Probabilistic Inference in a Dynamically Changing Environment , 2014, PLoS Comput. Biol..

[39]  Johannes Schemmel,et al.  Characterization and Compensation of Network-Level Anomalies in Mixed-Signal Neuromorphic Modeling Platforms , 2014, PloS one.