Bayesian Computation in Recurrent Neural Circuits

A large number of human psychophysical results have been successfully explained in recent years using Bayesian models. However, the neural implementation of such models remains largely unclear. In this article, we show that a network architecture commonly used to model the cerebral cortex can implement Bayesian inference for an arbitrary hidden Markov model. We illustrate the approach using an orientation discrimination task and a visual motion detection task. In the case of orientation discrimination, we show that the model network can infer the posterior distribution over orientations and correctly estimate stimulus orientation in the presence of significant noise. In the case of motion detection, we show that the resulting model network exhibits direction selectivity and correctly computes the posterior probabilities over motion direction and position. When used to solve the well-known random dots motion discrimination task, the model generates responses that mimic the activities of evidence-accumulating neurons in cortical areas LIP and FEF. The framework we introduce posits a new interpretation of cortical activities in terms of log posterior probabilities of stimuli occurring in the natural world.

[1]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[2]  David J. Fleet,et al.  Velocity Likelihoods in Biological and Machine Vision , 2001 .

[3]  Alexandre Pouget,et al.  Probabilistic Interpretation of Population Codes , 1996, Neural Computation.

[4]  J. Schall,et al.  Neural Control of Voluntary Movement Initiation , 1996, Science.

[5]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[6]  R. Desimone,et al.  A neural mechanism for working and recognition memory in inferior temporal cortex. , 1991, Science.

[7]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[8]  C. H. Anderson,et al.  Unifying Perspectives on Neuronal Codes and Processing , 1996, ICANN.

[9]  Victor A. F. Lamme,et al.  Contextual Modulation in Primary Visual Cortex , 1996, The Journal of Neuroscience.

[10]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[11]  J. Movshon,et al.  The analysis of visual motion: a comparison of neuronal and psychophysical performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[12]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[13]  H B Barlow,et al.  PATTERN RECOGNITION AND THE RESPONSES OF SENSORY NEURONS * , 1969, Annals of the New York Academy of Sciences.

[14]  Terrence J. Sejnowski,et al.  Bayesian Unsupervised Learning of Higher Order Structure , 1996, NIPS.

[15]  H S Seung,et al.  How the brain keeps the eyes still. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[16]  Rajesh P. N. Rao,et al.  Probabilistic Models of the Brain: Perception and Neural Function , 2002 .

[17]  I. Ohzawa,et al.  Organization of suppression in receptive fields of neurons in cat visual cortex. , 1992, Journal of neurophysiology.

[18]  James L. McClelland,et al.  The time course of perceptual choice: the leaky, competing accumulator model. , 2001, Psychological review.

[19]  Rajesh P. N. Rao,et al.  Dynamic Model of Visual Recognition Predicts Neural Response Properties in the Visual Cortex , 1997, Neural Computation.

[20]  Edward H. Adelson,et al.  Motion illusions as optimal percepts , 2002, Nature Neuroscience.

[21]  Geoffrey E. Hinton,et al.  Generative models for discovering sparse distributed representations. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[22]  A. Borst Seeing smells: imaging olfactory learning in bees , 1999, Nature Neuroscience.

[23]  P. Sajda,et al.  Inferring figure-ground using a recurrent integrate-and-fire neural circuit , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[24]  Ilya Nemenman,et al.  Fluctuation-Dissipation Theorem and Models of Learning , 2004, Neural Computation.

[25]  Bartlett W. Mel Synaptic integration in an excitable dendritic tree. , 1993, Journal of neurophysiology.

[26]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[27]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[28]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[29]  Si Wu,et al.  Computing with Continuous Attractors: Stability and Online Aspects , 2005, Neural Computation.

[30]  Peter E. Latham,et al.  Statistically Efficient Estimation Using Population Coding , 1998, Neural Computation.

[31]  P. Sajda,et al.  Inferring direction of figure using a recurrent integrate-and-fire neural circuit , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[32]  Geoffrey E. Hinton,et al.  Varieties of Helmholtz Machine , 1996, Neural Networks.

[33]  R. Ratcliff,et al.  Connectionist and diffusion models of reaction time. , 1999, Psychological review.

[34]  R. Duncan Luce,et al.  Response Times: Their Role in Inferring Elementary Mental Organization , 1986 .

[35]  John S. Bridle,et al.  Alpha-nets: A recurrent 'neural' network architecture with a hidden Markov model interpretation , 1990, Speech Commun..

[36]  Xiao-Jing Wang Synaptic reverberation underlying mnemonic persistent activity , 2001, Trends in Neurosciences.

[37]  R. Carpenter,et al.  The influence of urgency on decision time , 2000, Nature Neuroscience.

[38]  Paul Sajda,et al.  Integration of form and motion within a generative model of visual cortex , 2004, Neural Networks.

[39]  M. Shadlen,et al.  Response of Neurons in the Lateral Intraparietal Area during a Combined Visual Discrimination Reaction Time Task , 2002, The Journal of Neuroscience.

[40]  Christof Koch,et al.  Biophysics of Computation: Information Processing in Single Neurons (Computational Neuroscience Series) , 1998 .

[41]  H B Barlow,et al.  Single units and sensation: a neuron doctrine for perceptual psychology? , 1972, Perception.

[42]  J. Gold,et al.  Neural computations that underlie decisions about sensory stimuli , 2001, Trends in Cognitive Sciences.

[43]  Jianfeng Feng,et al.  Population approach to a neural discrimination task , 2006, Biological Cybernetics.

[44]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[45]  Si Wu,et al.  Sequential Bayesian Decoding with a Population of Neurons , 2003, Neural Computation.

[46]  A. Pouget,et al.  Reading population codes: a neural implementation of ideal observers , 1999, Nature Neuroscience.

[47]  Thomas J. Anastasio,et al.  Using Bayes' Rule to Model Multisensory Enhancement in the Superior Colliculus , 2000, Neural Computation.

[48]  Learning top-down gain control of feature selectivity in a recurrent network model of a visual cortical area , 2005, Vision Research.

[49]  Chris Eliasmith,et al.  Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems , 2004, IEEE Transactions on Neural Networks.

[50]  Eero P. Simoncelli Distributed representation and analysis of visual motion , 1993 .

[51]  Peter Dayan,et al.  Distributional Population Codes and Multiple Motion Models , 1998, NIPS.

[52]  Geoffrey E. Hinton,et al.  OPTIMAL PERCEPTUAL INFERENCE , 1983 .

[53]  M. Landy,et al.  Bayesian Modelling of Visual Perception , 2002 .

[54]  A. Hurlbert,et al.  Perception of three-dimensional shape influences colour perception through mutual illumination , 1999, Nature.

[55]  B L McNaughton,et al.  Interpreting neuronal population activity by reconstruction: unified framework with application to hippocampal place cells. , 1998, Journal of neurophysiology.

[56]  David Ascher,et al.  A Bayesian model for the measurement of visual velocity , 2000, Vision Research.

[57]  R. H. S. Carpenter,et al.  Neural computation of log likelihood in control of saccadic eye movements , 1995, Nature.

[58]  Michael I. Jordan,et al.  Graphical models: Probabilistic inference , 2002 .

[59]  Rajesh P. N. Rao,et al.  Bayesian Inference and Attentional Modulation in the Visual Cortex Correspondence and Requests for Reprints to Rajesh , 2005 .

[60]  Jeffrey D. Schall,et al.  Neural mechanisms of selection and control of visually guided eye movements , 1998, Neural Networks.

[61]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[62]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[63]  Vivien A. Casagrande,et al.  Biophysics of Computation: Information Processing in Single Neurons , 1999 .

[64]  Xiao-Jing Wang,et al.  Probabilistic Decision Making by Slow Reverberation in Cortical Circuits , 2002, Neuron.

[65]  Michael L. Platt,et al.  Neural correlates of decision variables in parietal cortex , 1999, Nature.

[66]  Paul Glimcher,et al.  Decisions, Decisions, Decisions Choosing a Biological Science of Choice , 2002, Neuron.

[67]  David Welch,et al.  Decisions, Decisions , 2001 .

[68]  R. Desimone,et al.  Competitive Mechanisms Subserve Attention in Macaque Areas V2 and V4 , 1999, The Journal of Neuroscience.

[69]  S. Laughlin,et al.  Predictive coding: a fresh view of inhibition in the retina , 1982, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[70]  D. V. van Essen,et al.  Neuronal responses to static texture patterns in area V1 of the alert macaque monkey. , 1992, Journal of neurophysiology.

[71]  P. Földiák,et al.  Forming sparse representations by local anti-Hebbian learning , 1990, Biological Cybernetics.

[72]  A. Diederich,et al.  Why aren’t all deep superior colliculus neurons multisensory? A Bayes’ ratio analysis , 2004, Cognitive, affective & behavioral neuroscience.

[73]  Rajesh P. N. Rao,et al.  An optimal estimation approach to visual perception and learning , 1999, Vision Research.

[74]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[75]  Aapo Hyvärinen,et al.  Interpreting Neural Response Variability as Monte Carlo Sampling of the Posterior , 2002, NIPS.

[76]  Yves Burnod,et al.  Bayesian inference in populations of cortical neurons: a model of motion integration and segmentation in area MT , 1999, Biological Cybernetics.

[77]  E. Bullmore,et al.  Society for Neuroscience Abstracts , 1997 .

[78]  J. Schall,et al.  Neural selection and control of visually guided eye movements. , 1999, Annual review of neuroscience.