Revealing nonlinear neural decoding by analyzing choices

Sensory data about most natural task-relevant variables are entangled with task-irrelevant nuisance variables. The neurons that encode these relevant signals typically constitute a nonlinear population code. Here we present a theoretical framework for quantifying how the brain uses or decodes its nonlinear information. Our theory obeys fundamental mathematical limitations on information content inherited from the sensory periphery, identifying redundant codes when there are many more cortical neurons than primary sensory neurons. The theory predicts that if the brain uses its nonlinear population codes optimally, then more informative patterns should be more correlated with choices. More specifically, the theory predicts a simple, easily computed quantitative relationship between fluctuating neural activity and behavioral choices that reveals the decoding efficiency. We analyze recordings from primary visual cortex of monkeys discriminating the distribution from which oriented stimuli were drawn, and find these data are consistent with the hypothesis of near-optimal nonlinear decoding.

[1]  Ha Hong,et al.  Performance-optimized hierarchical models predict neural responses in higher visual cortex , 2014, Proceedings of the National Academy of Sciences.

[2]  Stefan Treue,et al.  Feature-based attention influences motion processing gain in macaque visual cortex , 1999, Nature.

[3]  Haim Sompolinsky,et al.  Implications of Neuronal Diversity on Population Coding , 2006, Neural Computation.

[4]  John H R Maunsell,et al.  Potential confounds in estimating trial-to-trial correlations between neuronal response and behavior using choice probabilities. , 2012, Journal of neurophysiology.

[5]  Alexandre Pouget,et al.  Measuring Fisher Information Accurately in Correlated Neural Populations , 2015, PLoS Comput. Biol..

[6]  A. Pouget,et al.  Information-limiting correlations , 2014, Nature Neuroscience.

[7]  W. Newsome,et al.  Estimates of the Contribution of Single Neurons to Perception Depend on Timescale and Noise Correlation , 2009, The Journal of Neuroscience.

[8]  Matthias Bethge,et al.  Evaluating neuronal codes for inference using Fisher information , 2010, NIPS.

[9]  Bruce G. Cumming,et al.  Feedback Dynamics Determine the Structure of Spike-Count Correlation in Visual Cortex , 2016 .

[10]  J. Maunsell,et al.  Attention improves performance primarily by reducing interneuronal correlations , 2009, Nature Neuroscience.

[11]  Ehud Zohary,et al.  Correlated neuronal discharge rate and its implications for psychophysical performance , 1994, Nature.

[12]  T. Poggio,et al.  Synapses that compute motion. , 1987, Scientific American.

[13]  Ifije E. Ohiorhenuan,et al.  Sparse coding and high-order correlations in fine-scale cortical networks , 2010, Nature.

[14]  G. DeAngelis,et al.  Neural correlates of multisensory cue integration in macaque MSTd , 2008, Nature Neuroscience.

[15]  A. Parker,et al.  Perceptually Bistable Three-Dimensional Figures Evoke High Choice Probabilities in Cortical Area MT , 2001, The Journal of Neuroscience.

[16]  Daeyeol Lee,et al.  Effects of noise correlations on information encoding and decoding. , 2006, Journal of neurophysiology.

[17]  Anthony J. Movshon,et al.  Optimal representation of sensory information by neural populations , 2006, Nature Neuroscience.

[18]  Alexander S. Ecker,et al.  Decorrelated Neuronal Firing in Cortical Microcircuits , 2010, Science.

[19]  C. Salzman,et al.  Abstract Context Representations in Primate Amygdala and Prefrontal Cortex , 2015, Neuron.

[20]  Peter Dayan,et al.  The Effect of Correlated Variability on the Accuracy of a Population Code , 1999, Neural Computation.

[21]  W. Newsome,et al.  Context-dependent computation by recurrent dynamics in prefrontal cortex , 2013, Nature.

[22]  Doris Y. Tsao,et al.  Intelligent Information Loss: The Coding of Facial Identity, Head Pose, and Non-Face Information in the Macaque Face Patch System , 2015, The Journal of Neuroscience.

[23]  Alexander S. Ecker,et al.  State Dependence of Noise Correlations in Macaque Primary Visual Cortex , 2014, Neuron.

[24]  M. Shadlen,et al.  Response of Neurons in the Lateral Intraparietal Area during a Combined Visual Discrimination Reaction Time Task , 2002, The Journal of Neuroscience.

[25]  Alexander S. Ecker,et al.  Reassessing optimal neural population codes with neurometric functions , 2011, Proceedings of the National Academy of Sciences.

[26]  Gustavo Deco,et al.  Stimulus-dependent variability and noise correlations in cortical MT neurons , 2013, Proceedings of the National Academy of Sciences.

[27]  Alexander S. Ecker,et al.  Attentional fluctuations induce shared variability in macaque primary visual cortex , 2017, Nature Communications.

[28]  Xaq Pitkow,et al.  Inference in the Brain: Statistics Flowing in Redundant Population Codes , 2017, Neuron.

[29]  A. Tolias,et al.  Trial-to-trial, uncertainty-based adjustment of decision boundaries in visual categorization , 2013, Proceedings of the National Academy of Sciences.

[30]  Matthias Bethge,et al.  Optimal Short-Term Population Coding: When Fisher Information Fails , 2002, Neural Computation.

[31]  Arnulf B. A. Graf,et al.  Decoding the activity of neuronal populations in macaque primary visual cortex , 2011, Nature Neuroscience.

[32]  Richard D. Lange,et al.  Inferring the brain’s internal model from sensory responses in a probabilistic inference framework , 2016, bioRxiv.

[33]  A. Pouget,et al.  Correlations and Neuronal Population Information. , 2016, Annual review of neuroscience.

[34]  Valentin Dragoi,et al.  Adaptive coding of visual information in neural populations , 2008, Nature.

[35]  W. Newsome,et al.  Context-Dependent Changes in Functional Circuitry in Visual Area MT , 2008, Neuron.

[36]  R. Romo,et al.  Neuronal correlates of subjective sensory experience , 2005, Nature Neuroscience.

[37]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[38]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[39]  Haim Sompolinsky,et al.  Nonlinear Population Codes , 2004, Neural Computation.

[40]  A. Pouget,et al.  Behavior and neural basis of near-optimal visual search , 2011, Nature Neuroscience.

[41]  Nicole C Rust,et al.  Dynamic Target Match Signals in Perirhinal Cortex Can Be Explained by Instantaneous Computations That Act on Dynamic Input from Inferotemporal Cortex , 2014, The Journal of Neuroscience.

[42]  David D. Cox,et al.  Opinion TRENDS in Cognitive Sciences Vol.11 No.8 Untangling invariant object recognition , 2022 .

[43]  D. Bradley,et al.  Neural population code for fine perceptual decisions in area MT , 2005, Nature Neuroscience.

[44]  Alexandre Pouget,et al.  Origin of information-limiting noise correlations , 2015, Proceedings of the National Academy of Sciences.

[45]  Alexandre Pouget,et al.  Insights from a Simple Expression for Linear Fisher Information in a Recurrently Connected Population of Spiking Neurons , 2011, Neural Computation.

[46]  B. Cumming,et al.  Psychophysically measured task strategy for disparity discrimination is reflected in V2 neurons , 2007, Nature Neuroscience.

[47]  Nikolaus Kriegeskorte,et al.  Recurrence is required to capture the representational dynamics of the human visual system , 2019, Proceedings of the National Academy of Sciences.

[48]  A. Pouget,et al.  Not Noisy, Just Wrong: The Role of Suboptimal Inference in Behavioral Variability , 2012, Neuron.

[49]  Nicole C. Rust,et al.  Signals in inferotemporal and perirhinal cortex suggest an “untangling” of visual target information , 2013, Nature Neuroscience.

[50]  H. Sompolinsky,et al.  Sparseness and Expansion in Sensory Representations , 2014, Neuron.

[51]  G. DeAngelis,et al.  How Can Single Sensory Neurons Predict Behavior? , 2015, Neuron.

[52]  K. A. Davis,et al.  Auditory Processing of Spectral Cues for Sound Localization in the Inferior Colliculus , 2003, Journal of the Association for Research in Otolaryngology.

[53]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[54]  Eero P. Simoncelli,et al.  Neural Quadratic Discriminant Analysis: Nonlinear Decoding with V1-Like Computation , 2016, Neural Computation.

[55]  M. Bethge,et al.  Inferring decoding strategies from choice probabilities in the presence of correlated variability , 2013, Nature Neuroscience.

[56]  József Fiser,et al.  Perceptual Decision-Making as Probabilistic Inference by Neural Sampling , 2014, Neuron.

[57]  M. A. Smith,et al.  Stimulus Dependence of Neuronal Correlation in Primary Visual Cortex of the Macaque , 2005, The Journal of Neuroscience.

[58]  Ian Krajbich,et al.  Visual fixations and the computation and comparison of value in simple choice , 2010, Nature Neuroscience.

[59]  Nicole C. Rust,et al.  Selectivity and Tolerance (“Invariance”) Both Increase as Visual Information Propagates from Cortical Area V4 to IT , 2010, The Journal of Neuroscience.

[60]  Alexander S. Ecker,et al.  On the Structure of Neuronal Population Activity under Fluctuations in Attentional State , 2015, The Journal of Neuroscience.

[61]  Alexandre Pouget,et al.  Inferring decoding strategies for multiple correlated neural populations , 2017, bioRxiv.

[62]  H. Sompolinsky,et al.  Population coding in neuronal systems with correlated noise. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[63]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[64]  J. Donoghue,et al.  Neuronal Interactions Improve Cortical Population Coding of Movement Direction , 1999, The Journal of Neuroscience.

[65]  A. Pouget,et al.  Neural correlations, population coding and computation , 2006, Nature Reviews Neuroscience.

[66]  TJ Gawne,et al.  How independent are the messages carried by adjacent inferior temporal cortical neurons? , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[67]  K. H. Britten,et al.  A relationship between behavioral choice and the visual responses of neurons in macaque MT , 1996, Visual Neuroscience.

[68]  Wei Ji Ma,et al.  A neural basis of probabilistic computation in visual cortex , 2018, Nature Neuroscience.

[69]  Alexander S. Ecker,et al.  Recording chronically from the same neurons in awake, behaving primates. , 2007, Journal of neurophysiology.

[70]  M. Cohen,et al.  Measuring and interpreting neuronal correlations , 2011, Nature Neuroscience.

[71]  Bruce G. Cumming,et al.  Feedback Determines the Structure of Correlated Variability in Visual Cortex , 2016 .

[72]  Johannes Burge,et al.  Accuracy Maximization Analysis for Sensory-Perceptual Tasks: Computational Improvements, Filter Robustness, and Coding Advantages for Scaled Additive Noise , 2017, PLoS Comput. Biol..

[73]  J. Movshon,et al.  A computational analysis of the relationship between neuronal and behavioral responses to visual motion , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[74]  Alexander S. Ecker,et al.  The Effect of Noise Correlations in Populations of Diversely Tuned Neurons , 2011, The Journal of Neuroscience.

[75]  Xiao-Jing Wang,et al.  The importance of mixed selectivity in complex cognitive tasks , 2013, Nature.

[76]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[77]  M. Paradiso,et al.  A theory for the use of visual orientation information which exploits the columnar structure of striate cortex , 2004, Biological Cybernetics.