Cognition as a Window into Neuronal Population Space.

Understanding how cognitive processes affect the responses of sensory neurons may clarify the relationship between neuronal population activity and behavior. However, tools for analyzing neuronal activity have not kept up with technological advances in recording from large neuronal populations. Here, we describe prevalent hypotheses of how cognitive processes affect sensory neurons, driven largely by a model based on the activity of single neurons or pools of neurons as the units of computation. We then use simple simulations to expand this model to a new conceptual framework that focuses on subspaces of population activity as the relevant units of computation, uses comparisons between brain areas or to behavior to guide analyses of these subspaces, and suggests that population activity is optimized to decode the large variety of stimuli and tasks that animals encounter in natural behavior. This framework provides new ways of understanding the ever-growing quantity of recorded population activity data.

[1]  J. Reynolds,et al.  Attentional modulation of visual processing. , 2004, Annual review of neuroscience.

[2]  J. Maunsell,et al.  Graded Neuronal Modulations Related to Visual Spatial Attention , 2016, The Journal of Neuroscience.

[3]  Douglas A Ruff,et al.  Stimulus Dependence of Correlated Variability across Cortical Areas , 2016, The Journal of Neuroscience.

[4]  B. Cumming,et al.  Decision-Related Activity in Macaque V2 for Fine Disparity Discrimination Is Not Compatible with Optimal Linear Readout , 2017, The Journal of Neuroscience.

[5]  T. Womelsdorf,et al.  The role of neuronal synchronization in selective attention , 2007, Current Opinion in Neurobiology.

[6]  Katherine M. Armstrong,et al.  Selective gating of visual signals by microstimulation of frontal cortex , 2003, Nature.

[7]  Robert Desimone,et al.  Parallel and Serial Neural Mechanisms for Visual Search in Macaque Area V4 , 2005, Science.

[8]  Jude F. Mitchell,et al.  Differential Attention-Dependent Response Modulation across Cell Classes in Macaque Visual Area V4 , 2007, Neuron.

[9]  Si Wu,et al.  Perceptual training continuously refines neuronal population codes in primary visual cortex , 2014, Nature Neuroscience.

[10]  B. Cumming,et al.  Decision-related activity in sensory neurons reflects more than a neuron’s causal effect , 2009, Nature.

[11]  J. Serences,et al.  Attention Improves Transfer of Motion Information between V1 and MT , 2014, The Journal of Neuroscience.

[12]  R. Desimone,et al.  Gamma-band synchronization in visual cortex predicts speed of change detection , 2006, Nature.

[13]  M. A. Smith,et al.  Spatial and Temporal Scales of Neuronal Correlation in Primary Visual Cortex , 2008, The Journal of Neuroscience.

[14]  C. Law,et al.  Neural correlates of perceptual learning in a sensory-motor, but not a sensory, cortical area , 2008, Nature Neuroscience.

[15]  Gamaleldin F. Elsayed,et al.  Structure in neural population recordings: an expected byproduct of simpler phenomena? , 2017, Nature Neuroscience.

[16]  P. Roelfsema,et al.  Distinct Feedforward and Feedback Effects of Microstimulation in Visual Cortex Reveal Neural Mechanisms of Texture Segregation , 2017, Neuron.

[17]  Christopher C. Pack,et al.  The Contribution of Area MT to Visual Motion Perception Depends on Training , 2016, Neuron.

[18]  R. Quiroga,et al.  Extracting information from neuronal populations : information theory and decoding approaches , 2022 .

[19]  Alexander S. Ecker,et al.  On the Structure of Neuronal Population Activity under Fluctuations in Attentional State , 2015, The Journal of Neuroscience.

[20]  Pieter R. Roelfsema,et al.  Texture Segregation Causes Early Figure Enhancement and Later Ground Suppression in Areas V1 and V4 of Visual Cortex , 2016, Cerebral cortex.

[21]  Matthew T. Kaufman,et al.  Supplementary materials for : Cortical activity in the null space : permitting preparation without movement , 2014 .

[22]  Matthew T. Kaufman,et al.  Neural population dynamics during reaching , 2012, Nature.

[23]  Alexander Thiele,et al.  Attention-Induced Variance and Noise Correlation Reduction in Macaque V1 Is Mediated by NMDA Receptors , 2013, Neuron.

[24]  John P. Cunningham,et al.  Reorganization between preparatory and movement population responses in motor cortex , 2016, Nature Communications.

[25]  S. Yantis,et al.  Cortical mechanisms of space-based and object-based attentional control , 2003, Current Opinion in Neurobiology.

[26]  D. Knill Robust cue integration: a Bayesian model and evidence from cue-conflict studies with stereoscopic and figure cues to slant. , 2007, Journal of vision.

[27]  Xaq Pitkow,et al.  Inference in the Brain: Statistics Flowing in Redundant Population Codes , 2017, Neuron.

[28]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[29]  Rava Azeredo da Silveira,et al.  High-Fidelity Coding with Correlated Neurons , 2013, PLoS Comput. Biol..

[30]  J. Maunsell,et al.  The role of attention in visual processing. , 2002, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[31]  Pieter R Roelfsema,et al.  Microstimulation of area V4 has little effect on spatial attention and on perception of phosphenes evoked in area V1. , 2015, Journal of neurophysiology.

[32]  Jude F. Mitchell,et al.  Spatial Attention Decorrelates Intrinsic Activity Fluctuations in Macaque Area V4 , 2009, Neuron.

[33]  K. H. Britten,et al.  A relationship between behavioral choice and the visual responses of neurons in macaque MT , 1996, Visual Neuroscience.

[34]  W. Martin Usrey,et al.  Attention Enhances Synaptic Efficacy and Signal-to-Noise in Neural Circuits , 2013, Nature.

[35]  Albert Compte,et al.  Sensory integration dynamics in a hierarchical network explains choice probabilities in cortical area MT , 2015, Nature Communications.

[36]  Byron M. Yu,et al.  Brain–computer interfaces for dissecting cognitive processes underlying sensorimotor control , 2016, Current Opinion in Neurobiology.

[37]  Chandramouli Chandrasekaran,et al.  Computational principles and models of multisensory integration , 2017, Current Opinion in Neurobiology.

[38]  M. Carandini,et al.  Probing perceptual decisions in rodents , 2013, Nature Neuroscience.

[39]  R. Yuste From the neuron doctrine to neural networks , 2015, Nature Reviews Neuroscience.

[40]  J. Maunsell,et al.  Attention improves performance primarily by reducing interneuronal correlations , 2009, Nature Neuroscience.

[41]  Ehud Zohary,et al.  Correlated neuronal discharge rate and its implications for psychophysical performance , 1994, Nature.

[42]  Timothy D. Hanks,et al.  Neural underpinnings of the evidence accumulator , 2016, Current Opinion in Neurobiology.

[43]  G. Orban,et al.  Learning to See the Difference Specifically Alters the Most Informative V4 Neurons , 2006, The Journal of Neuroscience.

[44]  Douglas A Ruff,et al.  Attention Increases Spike Count Correlations between Visual Cortical Areas , 2016, The Journal of Neuroscience.

[45]  Timothy Q. Gentner,et al.  Associative Learning Enhances Population Coding by Inverting Interneuronal Correlation Patterns , 2013, Neuron.

[46]  Ha Hong,et al.  Performance-optimized hierarchical models predict neural responses in higher visual cortex , 2014, Proceedings of the National Academy of Sciences.

[47]  P. Roelfsema,et al.  Simultaneous selection by object-based attention in visual and frontal cortex , 2014, Proceedings of the National Academy of Sciences.

[48]  J. Maunsell,et al.  Attention-related changes in correlated neuronal activity arise from normalization mechanisms , 2017, Nature Neuroscience.

[49]  A. Pouget,et al.  Information-limiting correlations , 2014, Nature Neuroscience.

[50]  Earl K Miller,et al.  Cortical circuits for the control of attention , 2012, Current Opinion in Neurobiology.

[51]  Marisa Carrasco,et al.  Attentional enhancement of spatial resolution: linking behavioural and neurophysiological evidence , 2013, Nature Reviews Neuroscience.

[52]  Robert Desimone,et al.  Lesions of prefrontal cortex reduce attentional modulation of neuronal responses and synchrony in V4 , 2014, Nature Neuroscience.

[53]  Nicole C. Rust,et al.  Signals in inferotemporal and perirhinal cortex suggest an “untangling” of visual target information , 2013, Nature Neuroscience.

[54]  Bruce G Cumming,et al.  Decision-related activity in sensory neurons: correlations among neurons and with behavior. , 2012, Annual review of neuroscience.

[55]  P. Fries Rhythms for Cognition: Communication through Coherence , 2015, Neuron.

[56]  G. DeAngelis,et al.  Fine Discrimination Training Alters the Causal Contribution of Macaque Area MT to Depth Perception , 2008, Neuron.

[57]  Eero P. Simoncelli,et al.  Partitioning neuronal variability , 2014, Nature Neuroscience.

[58]  Douglas A Ruff,et al.  Attention can increase or decrease spike count correlations between pairs of neurons depending on their role in a task , 2014, Nature Neuroscience.

[59]  Alexandre Zénon,et al.  Attention deficits without cortical neuronal deficits , 2012, Nature.

[60]  Stefan Everling,et al.  Interareal Spike-Train Correlations of Anterior Cingulate and Dorsal Prefrontal Cortex during Attention Shifts , 2015, The Journal of Neuroscience.

[61]  Xin Huang,et al.  Noise correlations in cortical area MT and their potential impact on trial-by-trial variation in the direction and speed of smooth-pursuit eye movements. , 2009, Journal of neurophysiology.

[62]  Eero P. Simoncelli,et al.  Attention stabilizes the shared gain of V4 populations , 2015, eLife.

[63]  Bruce G Cumming,et al.  Feedforward and Feedback Sources of Choice Probability in Neural Population Responses This Review Comes from a Themed Issue on Neurobiology of Cognitive Behavior Evidence for Feed-forward Models and Optimal Linear Readout? , 2022 .

[64]  Carrie J. McAdams,et al.  Effects of Attention on Orientation-Tuning Functions of Single Neurons in Macaque Cortical Area V4 , 1999, The Journal of Neuroscience.

[65]  Alexandre Pouget,et al.  Measuring Fisher Information Accurately in Correlated Neural Populations , 2015, PLoS Comput. Biol..

[66]  M. Cohen,et al.  Relating normalization to neuronal populations across cortical areas. , 2016, Journal of neurophysiology.

[67]  Alexandre Pouget,et al.  Optimal multisensory decision-making in a reaction-time task , 2014, eLife.

[68]  M. Cohen,et al.  Measuring and interpreting neuronal correlations , 2011, Nature Neuroscience.

[69]  Marc A Sommer,et al.  Spatial and Temporal Scales of Neuronal Correlation in Visual Area V4 , 2013, The Journal of Neuroscience.

[70]  J. Maunsell,et al.  Effects of task difficulty and target likelihood in area V4 of macaque monkeys. , 2006, Journal of neurophysiology.

[71]  Thomas Zhihao Luo,et al.  Neuronal Modulations in Visual Cortex Are Associated with Only One of Multiple Components of Attention , 2015, Neuron.

[72]  J. DiCarlo,et al.  Using goal-driven deep learning models to understand sensory cortex , 2016, Nature Neuroscience.

[73]  John T. Serences,et al.  Visual attention mitigates information loss in small- and large-scale neural codes , 2015, Trends in Cognitive Sciences.

[74]  E. Miller,et al.  Top-Down Versus Bottom-Up Control of Attention in the Prefrontal and Posterior Parietal Cortices , 2007, Science.

[75]  Alexandre Pouget,et al.  Origin of information-limiting noise correlations , 2015, Proceedings of the National Academy of Sciences.

[76]  John H. Reynolds,et al.  Laminar Organization of Attentional Modulation in Macaque Visual Area V4 , 2017, Neuron.

[77]  Daniel L. K. Yamins,et al.  Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition , 2014, PLoS Comput. Biol..

[78]  Nikolaus Kriegeskorte,et al.  Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation , 2014, PLoS Comput. Biol..

[79]  Mehrdad Jazayeri,et al.  Navigating the Neural Space in Search of the Neural Code , 2017, Neuron.

[80]  J. Maunsell,et al.  Using Neuronal Populations to Study the Mechanisms Underlying Spatial and Feature Attention , 2011, Neuron.

[81]  A. Pouget,et al.  Neural correlations, population coding and computation , 2006, Nature Reviews Neuroscience.

[82]  John P. Cunningham,et al.  Behaviorally Selective Engagement of Short-Latency Effector Pathways by Motor Cortex , 2017, Neuron.

[83]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[84]  T. Womelsdorf,et al.  Attentional Stimulus Selection through Selective Synchronization between Monkey Visual Areas , 2012, Neuron.

[85]  D. Bradley,et al.  Structure and function of visual area MT. , 2005, Annual review of neuroscience.

[86]  Douglas A Ruff,et al.  Global Cognitive Factors Modulate Correlated Response Variability between V4 Neurons , 2014, The Journal of Neuroscience.

[87]  R. Desimone,et al.  High-Frequency, Long-Range Coupling Between Prefrontal and Visual Cortex During Attention , 2009, Science.

[88]  Yong Gu,et al.  Perceptual Learning Reduces Interneuronal Correlations in Macaque Visual Cortex , 2011, Neuron.

[89]  Byron M. Yu,et al.  Dimensionality reduction for large-scale neural recordings , 2014, Nature Neuroscience.

[90]  Ivan N Pigarev,et al.  Neural Mechanisms of Visual Attention: How Top-Down Feedback Highlights Relevant Locations , 2007, Science.

[91]  Konrad Paul Kording,et al.  Review TRENDS in Cognitive Sciences Vol.10 No.7 July 2006 Special Issue: Probabilistic models of cognition Bayesian decision theory in sensorimotor control , 2022 .

[92]  R. Jacobs,et al.  Optimal integration of texture and motion cues to depth , 1999, Vision Research.

[93]  Dragan F. Dimitrov,et al.  Reversible large-scale modification of cortical networks during neuroprosthetic control , 2011, Nature Neuroscience.

[94]  J. Movshon,et al.  The analysis of visual motion: a comparison of neuronal and psychophysical performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[95]  W. Newsome,et al.  Context-Dependent Changes in Functional Circuitry in Visual Area MT , 2008, Neuron.

[96]  R. Desimone,et al.  Modulation of Oscillatory Neuronal Synchronization by Selective Visual Attention , 2001, Science.

[97]  Marc H Schieber,et al.  Rapid acquisition of novel interface control by small ensembles of arbitrarily selected primary motor cortex neurons. , 2014, Journal of neurophysiology.

[98]  G. DeAngelis,et al.  How Can Single Sensory Neurons Predict Behavior? , 2015, Neuron.

[99]  Byron M. Yu,et al.  Neural constraints on learning , 2014, Nature.

[100]  Brent Doiron,et al.  Attentional modulation of neuronal variability in circuit models of cortex , 2017, eLife.

[101]  R. Desimone,et al.  Local precision of visuotopic organization in the middle temporal area (MT) of the macaque , 2004, Experimental Brain Research.

[102]  Leslie G. Ungerleider,et al.  The neural systems that mediate human perceptual decision making , 2008, Nature Reviews Neuroscience.

[103]  W. Newsome,et al.  Estimates of the Contribution of Single Neurons to Perception Depend on Timescale and Noise Correlation , 2009, The Journal of Neuroscience.

[104]  W. Bair,et al.  Correlated Firing in Macaque Visual Area MT: Time Scales and Relationship to Behavior , 2001, The Journal of Neuroscience.

[105]  John W Morley,et al.  Local and Global Correlations between Neurons in the Middle Temporal Area of Primate Visual Cortex. , 2015, Cerebral cortex.

[106]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[107]  A. Pouget,et al.  Correlations and Neuronal Population Information. , 2016, Annual review of neuroscience.

[108]  J. Maunsell,et al.  Feature-based attention in visual cortex , 2006, Trends in Neurosciences.

[109]  James J. DiCarlo,et al.  How Does the Brain Solve Visual Object Recognition? , 2012, Neuron.

[110]  Peter Dayan,et al.  The Effect of Correlated Variability on the Accuracy of a Population Code , 1999, Neural Computation.

[111]  C. Gilbert,et al.  Brain States: Top-Down Influences in Sensory Processing , 2007, Neuron.

[112]  A. Parker,et al.  Sense and the single neuron: probing the physiology of perception. , 1998, Annual review of neuroscience.

[113]  Douglas A Ruff,et al.  A normalization model suggests that attention changes the weighting of inputs between visual areas , 2017, Proceedings of the National Academy of Sciences.

[114]  Gregory C DeAngelis,et al.  Coding of horizontal disparity and velocity by MT neurons in the alert macaque. , 2003, Journal of neurophysiology.

[115]  Nikolaus Kriegeskorte,et al.  Relating Population-Code Representations between Man, Monkey, and Computational Models , 2009, Front. Neurosci..

[116]  J. Movshon,et al.  A computational analysis of the relationship between neuronal and behavioral responses to visual motion , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[117]  Christopher R Fetsch,et al.  Neural correlates of reliability-based cue weighting during multisensory integration , 2011, Nature Neuroscience.

[118]  David D. Cox,et al.  Opinion TRENDS in Cognitive Sciences Vol.11 No.8 Untangling invariant object recognition , 2022 .

[119]  M. A. Smith,et al.  Stimulus Dependence of Neuronal Correlation in Primary Visual Cortex of the Macaque , 2005, The Journal of Neuroscience.

[120]  G. Karmos,et al.  Entrainment of Neuronal Oscillations as a Mechanism of Attentional Selection , 2008, Science.

[121]  Richard T Born,et al.  Joint tuning for direction of motion and binocular disparity in macaque MT is largely separable. , 2013, Journal of neurophysiology.

[122]  John H. R. Maunsell,et al.  Neuronal Mechanisms of Visual Attention. , 2015, Annual review of vision science.