Inferring the brain’s internal model from sensory responses in a probabilistic inference framework

During perception, the brain combines information received from its senses with prior information about the world (von Helmholtz, 1867) – a process whose neural basis is still unclear. If sensory neurons represent posterior beliefs in a Bayesian inference process, then they, just like the beliefs themselves, must depend both on sensory inputs and on prior information. We derive predictions for how prior knowledge relates a neuron’s stimulus tuning to its response covariability in a way specific to the psychophysical task performed by the brain, and for how this covariability arises from both feedforward and feedback signals. We show that our predictions are in agreement with existing measurements. Finally, we demonstrate how to use neurophysiological measurements to reverse-engineer information about the subject’s internal beliefs about the structure of the task. Our results reinterpret neural covariability as signatures of Bayesian inference and provide new insights into their cause and their function.

[1]  H. Helmholtz Handbuch der physiologischen Optik , 2015 .

[2]  C. Law,et al.  Neural correlates of perceptual learning in a sensory-motor, but not a sensory, cortical area , 2008, Nature Neuroscience.

[3]  W. Newsome,et al.  Context-Dependent Changes in Functional Circuitry in Visual Area MT , 2008, Neuron.

[4]  Hideyuki Suzuki,et al.  Population Code Dynamics in Categorical Perception , 2016, Scientific Reports.

[5]  Alexander S. Ecker,et al.  On the structure of population activity under fluctuations in attentional state , 2015 .

[6]  Wolfgang Maass,et al.  Neural Dynamics as Sampling: A Model for Stochastic Computation in Recurrent Networks of Spiking Neurons , 2011, PLoS Comput. Biol..

[7]  Wei Ji Ma,et al.  Neural coding of uncertainty and probability. , 2014, Annual review of neuroscience.

[8]  G. DeAngelis,et al.  How Can Single Sensory Neurons Predict Behavior? , 2015, Neuron.

[9]  Wolfgang Maass,et al.  Probabilistic Inference in General Graphical Models through Sampling in Stochastic Networks of Spiking Neurons , 2011, PLoS Comput. Biol..

[10]  Peter Dayan,et al.  The Effect of Correlated Variability on the Accuracy of a Population Code , 1999, Neural Computation.

[11]  A. Parker,et al.  Sense and the single neuron: probing the physiology of perception. , 1998, Annual review of neuroscience.

[12]  R. Finke Levels of equivalence in imagery and perception. , 1980 .

[13]  Eero P. Simoncelli,et al.  Efficient Sensory Encoding and Bayesian Inference with Heterogeneous Neural Populations , 2014, Neural Computation.

[14]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[15]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[16]  Daniel D. Lee,et al.  Dynamic belief state representations , 2014, Current Opinion in Neurobiology.

[17]  Byron M. Yu,et al.  Dimensionality reduction for large-scale neural recordings , 2014, Nature Neuroscience.

[18]  Eero P. Simoncelli,et al.  Author response: Attention stabilizes the shared gain of V4 populations , 2015 .

[19]  Eero P. Simoncelli,et al.  Natural signal statistics and sensory gain control , 2001, Nature Neuroscience.

[20]  M. Bethge,et al.  Inferring decoding strategies from choice probabilities in the presence of correlated variability , 2013, Nature Neuroscience.

[21]  J. Movshon,et al.  The analysis of visual motion: a comparison of neuronal and psychophysical performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[22]  R. Finke,et al.  Principles of mental imagery , 1989 .

[23]  Yong Gu,et al.  Perceptual Learning Reduces Interneuronal Correlations in Macaque Visual Cortex , 2011, Neuron.

[24]  A. Faisal,et al.  Noise in the nervous system , 2008, Nature Reviews Neuroscience.

[25]  B. Cumming,et al.  Decision-related activity in sensory neurons reflects more than a neuron’s causal effect , 2009, Nature.

[26]  Alexander S. Ecker,et al.  On the Structure of Neuronal Population Activity under Fluctuations in Attentional State , 2015, The Journal of Neuroscience.

[27]  Edmund R S Kunji,et al.  Uncoupling protein 1 binds one nucleotide per monomer and is stabilized by tightly bound cardiolipin , 2015, Proceedings of the National Academy of Sciences.

[28]  Wei Ji Ma,et al.  Probabilistic brains: knowns and , 2013 .

[29]  A. Yuille,et al.  Object perception as Bayesian inference. , 2004, Annual review of psychology.

[30]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[31]  Maneesh Sahani,et al.  Learning stable, regularised latent models of neural population dynamics , 2012, Network.

[32]  K. H. Britten,et al.  A relationship between behavioral choice and the visual responses of neurons in macaque MT , 1996, Visual Neuroscience.

[33]  Bruce G Cumming,et al.  Decision-related activity in sensory neurons: correlations among neurons and with behavior. , 2012, Annual review of neuroscience.

[34]  Eero P. Simoncelli,et al.  Partitioning neuronal variability , 2014, Nature Neuroscience.

[35]  A. Yuille,et al.  Opinion TRENDS in Cognitive Sciences Vol.10 No.7 July 2006 Special Issue: Probabilistic models of cognition Vision as Bayesian inference: analysis by synthesis? , 2022 .

[36]  D. Whitney,et al.  Serial dependence in visual perception , 2011 .

[37]  Timothy D. Hanks,et al.  Probabilistic Population Codes for Bayesian Decision Making , 2008, Neuron.

[38]  T. Uka,et al.  Dynamic Readout of Behaviorally Relevant Signals from Area MT during Task Switching , 2009, Neuron.

[39]  P. Cz. Handbuch der physiologischen Optik , 1896 .

[40]  József Fiser,et al.  Perceptual Decision-Making as Probabilistic Inference by Neural Sampling , 2014, Neuron.

[41]  Eero P. Simoncelli,et al.  Noise characteristics and prior expectations in human visual speed perception , 2006, Nature Neuroscience.

[42]  C. Law,et al.  Reinforcement learning can account for associative and perceptual learning on a visual decision task , 2009, Nature Neuroscience.

[43]  Bruce G. Cumming,et al.  Decision-Related Activity in Sensory Neurons May Depend on the Columnar Architecture of Cerebral Cortex , 2014, The Journal of Neuroscience.

[44]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[45]  David Mumford,et al.  On the computational architecture of the neocortex , 2004, Biological Cybernetics.

[46]  A. Pouget,et al.  Marginalization in Neural Circuits with Divisive Normalization , 2011, The Journal of Neuroscience.

[47]  P. Berkes,et al.  Statistically Optimal Perception and Learning: from Behavior to Neural Representations , 2022 .

[48]  Cristina Savin,et al.  Spatio-temporal Representations of Uncertainty in Spiking Neural Networks , 2014, NIPS.

[49]  C. Summerfield,et al.  Expectation in perceptual decision making: neural and computational mechanisms , 2014, Nature Reviews Neuroscience.

[50]  A. Pouget,et al.  Neural correlations, population coding and computation , 2006, Nature Reviews Neuroscience.

[51]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[52]  D. Wolpert,et al.  Cognitive Tomography Reveals Complex, Task-Independent Mental Representations , 2013, Current Biology.

[53]  A. Pouget,et al.  Correlations and Neuronal Population Information. , 2016, Annual review of neuroscience.

[54]  Aapo Hyvärinen,et al.  Interpreting Neural Response Variability as Monte Carlo Sampling of the Posterior , 2002, NIPS.

[55]  A. Pouget,et al.  Probabilistic brains: knowns and unknowns , 2013, Nature Neuroscience.

[56]  József Fiser,et al.  Neural Variability and Sampling-Based Probabilistic Representations in the Visual Cortex , 2016, Neuron.

[57]  Konrad Paul Kording,et al.  Causal Inference in Multisensory Perception , 2007, PloS one.

[58]  Jakob H. Macke,et al.  Low-dimensional models of neural population activity in sensory cortical circuits , 2014, NIPS.

[59]  Daniel Chicharro,et al.  Decision-related signals in the presence of nonzero signal stimuli, internal bias, and feedback , 2017, bioRxiv.

[60]  Haim Sompolinsky,et al.  Implications of Neuronal Diversity on Population Coding , 2006, Neural Computation.

[61]  A. Pouget,et al.  Information-limiting correlations , 2014, Nature Neuroscience.

[62]  József Fiser,et al.  Spontaneous Cortical Activity Reveals Hallmarks of an Optimal Internal Model of the Environment , 2011, Science.

[63]  Jörg Lücke,et al.  Are V1 Simple Cells Optimized for Visual Occlusions? A Comparative Study , 2013, PLoS Comput. Biol..

[64]  R. von der Heydt,et al.  Illusory contours and cortical neuron responses. , 1984, Science.

[65]  Brent Doiron,et al.  The mechanics of state-dependent neural correlations , 2016, Nature Neuroscience.

[66]  P. Goldman-Rakic,et al.  Preface: Cerebral Cortex Has Come of Age , 1991 .

[67]  J. Movshon,et al.  A computational analysis of the relationship between neuronal and behavioral responses to visual motion , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[68]  Alexander S. Ecker,et al.  The Effect of Noise Correlations in Populations of Diversely Tuned Neurons , 2011, The Journal of Neuroscience.

[69]  P. Dayan,et al.  A mathematical model explains saturating axon guidance responses to molecular gradients , 2016, eLife.

[70]  Alexandre Pouget,et al.  Probabilistic Interpretation of Population Codes , 1996, Neural Computation.

[71]  Albert Compte,et al.  Sensory integration dynamics in a hierarchical network explains choice probabilities in cortical area MT , 2015, Nature Communications.

[72]  R. Romo,et al.  Touch and go: decision-making mechanisms in somatosensation. , 2001, Annual review of neuroscience.

[73]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[74]  C. Gilbert,et al.  Top-Down Modulation of Lateral Interactions in Visual Cortex , 2013, The Journal of Neuroscience.

[75]  Alexandre Pouget,et al.  Origin of information-limiting noise correlations , 2015, Proceedings of the National Academy of Sciences.

[76]  Laurence Aitchison,et al.  The Hamiltonian Brain: Efficient Probabilistic Inference with Excitatory-Inhibitory Neural Circuit Dynamics , 2014, PLoS Comput. Biol..

[77]  Bruce G. Cumming,et al.  Feedback Dynamics Determine the Structure of Spike-Count Correlation in Visual Cortex , 2016 .

[78]  Karen R Dobkins,et al.  The face inversion effect in infants is driven by high, and not low, spatial frequencies. , 2014, Journal of vision.

[79]  Ehud Zohary,et al.  Correlated neuronal discharge rate and its implications for psychophysical performance , 1994, Nature.

[80]  Xue-Xin Wei,et al.  A Bayesian observer model constrained by efficient coding can explain 'anti-Bayesian' percepts , 2015, Nature Neuroscience.

[81]  Alexander S. Ecker,et al.  State Dependence of Noise Correlations in Macaque Primary Visual Cortex , 2014, Neuron.

[82]  Bruce Cumming,et al.  Correlations between the activity of sensory neurons and behavior: how much do they tell us about a neuron's causality? , 2010, Current Opinion in Neurobiology.

[83]  Wieland Brendel,et al.  Demixed Principal Component Analysis , 2011, NIPS.

[84]  D. Perrett,et al.  The `Ideal Homunculus': decoding neural population signals , 1998, Trends in Neurosciences.

[85]  Alexander S. Ecker,et al.  On the structure of neuronal population activity under fluctuations in attentional state , 2015, bioRxiv.

[86]  W. Newsome,et al.  Estimates of the Contribution of Single Neurons to Perception Depend on Timescale and Noise Correlation , 2009, The Journal of Neuroscience.

[87]  Naoshige Uchida,et al.  Demixed principal component analysis of neural population data , 2014, eLife.

[88]  T. Hensch Critical period plasticity in local cortical circuits , 2005, Nature Reviews Neuroscience.

[89]  James J. DiCarlo,et al.  Unsupervised Natural Experience Rapidly Alters Invariant Object Representation in Visual Cortex , 2008, Science.

[90]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[91]  Eero P. Simoncelli,et al.  Attention stabilizes the shared gain of V4 populations , 2015, eLife.

[92]  Bruno A Olshausen,et al.  Sparse coding of sensory inputs , 2004, Current Opinion in Neurobiology.

[93]  Pieter R Roelfsema,et al.  Belief states as a framework to explain extra-retinal influences in visual cortex , 2015, Current Opinion in Neurobiology.

[94]  Richard D. Lange,et al.  Characterizing and interpreting the influence of internal variables on sensory activity , 2017, Current Opinion in Neurobiology.

[95]  Samuel J. Gershman,et al.  Complex Probabilistic Inference , 2017 .

[96]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[97]  Thomas D Albright,et al.  On the Perception of Probable Things: Neural Substrates of Associative Memory, Imagery, and Perception , 2012, Neuron.

[98]  Johannes Schemmel,et al.  Stochastic inference with spiking neurons in the high-conductance state , 2016, Physical review. E.

[99]  B. Cumming,et al.  Psychophysically measured task strategy for disparity discrimination is reflected in V2 neurons , 2007, Nature Neuroscience.

[100]  A. Pouget,et al.  Not Noisy, Just Wrong: The Role of Suboptimal Inference in Behavioral Variability , 2012, Neuron.