Encoding of continuous perceptual choices in human early visual cortex

Research on the neural mechanisms of perceptual decision-making has typically focused on simple categorical choices, say between two alternative motion directions. Studies on such discrete alternatives have often suggested that choices are encoded either in a motor-based or in an abstract, categorical format in regions beyond sensory cortex. However, many sensory features are graded rather than discrete, raising the question how choices are encoded when they span the full sensory continuum. Here we assessed this using motion stimuli that could vary anywhere between 0° and 360°. We employed a combination of neuroimaging and encoding models based on Gaussian Process Regression to assess how either stimuli or choices were encoded in brain responses. We found that single-voxel tuning patterns could be used to reconstruct the trial-by-trial physical direction of motion as well as the participants’ continuous choices. Importantly, these continuous choice signals were primarily observed in early visual areas. The tuning properties in this region generalized between choice encoding and stimulus encoding, even for reports that reflected pure guessing. We found only little information related to the decision outcome in regions beyond visual cortex, such as parietal cortex, possibly because our task did not involve differential motor preparation. This could suggest that decisions for continuous stimuli take can place already in sensory brain regions, potentially using similar mechanisms to the sensory recruitment in visual working memory.

[1]  Felix M. Töpfer,et al.  Psychophysics and computational modeling of feature-continuous motion perception , 2022, Journal of vision.

[2]  R. Goebel,et al.  The dual nature of the BOLD signal: Responses in visual area hMT+ reflect both input properties and perceptual decision , 2021, Human brain mapping.

[3]  Adam Kohn,et al.  Decision Signals in the Local Field Potentials of Early and Mid-Level Macaque Visual Cortex. , 2020, Cerebral cortex.

[4]  D. Rueckert,et al.  Heterogeneity in Brain Microstructural Development Following Preterm Birth , 2020, Cerebral cortex.

[5]  Florent Meyniel,et al.  Large-scale dynamics of perceptual decision information across human cortex , 2020, Nature Communications.

[6]  Jordy Thielen,et al.  Evidence for confounding eye movements under attempted fixation and active viewing in cognitive neuroscience , 2019, Scientific Reports.

[7]  Jörn Diedrichsen,et al.  Peeling the Onion of Brain Representations. , 2019, Annual review of neuroscience.

[8]  Justin L. Gardner,et al.  Inverted Encoding Models Reconstruct an Arbitrary Model Response, Not the Stimulus , 2019, eNeuro.

[9]  Steven J. Luck,et al.  Decoding motion direction using the topography of sustained ERPs and alpha oscillations , 2019, NeuroImage.

[10]  Roger Ratcliff,et al.  Decision Making on Spatially Continuous Scales , 2018, Psychological review.

[11]  Jordy Thielen,et al.  No evidence for confounding orientation-dependent fixational eye movements under baseline conditions , 2018, Scientific Reports.

[12]  Madhura Ketkar,et al.  Combined fMRI- and eye movement-based decoding of bistable plaid motion perception , 2018, NeuroImage.

[13]  Jens Schwarzbach,et al.  Decoding of auditory and tactile perceptual decisions in parietal cortex , 2017, NeuroImage.

[14]  E. Miller,et al.  Gradual progression from sensory to task-related processing in cerebral cortex , 2017, Proceedings of the National Academy of Sciences.

[15]  Anne E. Urai,et al.  Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias , 2017, Nature Communications.

[16]  Monica Z. Weiland,et al.  Gaussian Process Regression for Predictive But Interpretable Machine Learning Models: An Example of Predicting Mental Workload across Tasks , 2017, Front. Hum. Neurosci..

[17]  Timothy D. Hanks,et al.  Perceptual Decision Making in Rodents, Monkeys, and Humans , 2017, Neuron.

[18]  Stefano Fusi,et al.  Why neurons mix: high dimensionality for higher cognition , 2016, Current Opinion in Neurobiology.

[19]  Philip L. Smith,et al.  Diffusion theory of decision making in continuous report. , 2016, Psychological review.

[20]  J. Haynes A Primer on Pattern-Based Approaches to fMRI: Principles, Pitfalls, and Perspectives , 2015, Neuron.

[21]  Kingson Man,et al.  Multivariate cross-classification: applying machine learning techniques to characterize abstraction in neural representations , 2015, Front. Hum. Neurosci..

[22]  Jeremy Freeman,et al.  Motion Direction Biases and Decoding in Human Visual Cortex , 2014, The Journal of Neuroscience.

[23]  Radoslaw Martin Cichy,et al.  The Neural Code for Face Orientation in the Human Fusiform Face Area , 2014, The Journal of Neuroscience.

[24]  Il Memming Park,et al.  Encoding and decoding in parietal cortex during sensorimotor decision-making , 2014, Nature Neuroscience.

[25]  M. Corbetta,et al.  Decision and action planning signals in human posterior parietal cortex during delayed perceptual choices , 2014, The European journal of neuroscience.

[26]  Frank Tong,et al.  Spatial specificity of working memory representations in the early visual cortex. , 2014, Journal of vision.

[27]  Xiao-Jing Wang,et al.  The importance of mixed selectivity in complex cognitive tasks , 2013, Nature.

[28]  Jonathan D. Nelson,et al.  How Embodied Is Perceptual Decision Making? Evidence for Separate Processing of Perceptual and Motor Decisions , 2013, The Journal of Neuroscience.

[29]  Stefan Bode,et al.  Similar neural mechanisms for perceptual guesses and free decisions , 2013, NeuroImage.

[30]  M. A. Goodale,et al.  What is the best fixation target? The effect of target shape on stability of fixational eye movements , 2013, Vision Research.

[31]  Martin N. Hebart,et al.  Human visual and parietal cortex encode visual choices independent of motor plans , 2012, NeuroImage.

[32]  Alexander C. Huk,et al.  Neural correlates and neural computations in posterior parietal cortex during perceptual decision-making , 2012, Front. Integr. Neurosci..

[33]  A. Schleicher,et al.  Cytoarchitectonic mapping of the human dorsal extrastriate cortex , 2012, Brain Structure and Function.

[34]  Radoslaw Martin Cichy,et al.  Imagery and perception share cortical representations of content and location. , 2012, Cerebral Cortex.

[35]  Timothy J. Pleskac,et al.  Neural correlates of evidence accumulation in a perceptual decision task. , 2011, Journal of neurophysiology.

[36]  Jack L. Gallant,et al.  Encoding and decoding in fMRI , 2011, NeuroImage.

[37]  David J. Freedman,et al.  A proposed common neural mechanism for categorization and perceptual decisions , 2011, Nature Neuroscience.

[38]  A. Engel,et al.  Cortical Network Dynamics of Perceptual Decision-Making in the Human Brain , 2011, Frontiers in Human Neuroscience.

[39]  J. Gold,et al.  Distinct Representations of a Perceptual Decision and the Associated Oculomotor Plan in the Monkey Lateral Intraparietal Area , 2011, The Journal of Neuroscience.

[40]  J. Anthony Movshon,et al.  Visual response properties of striate cortical neurons projecting to V2 in macaque , 2010 .

[41]  Carl E. Rasmussen,et al.  Gaussian Processes for Machine Learning (GPML) Toolbox , 2010, J. Mach. Learn. Res..

[42]  Nikolaus Kriegeskorte,et al.  How does an fMRI voxel sample the neuronal activity pattern: Compact-kernel or complex spatiotemporal filter? , 2010, NeuroImage.

[43]  D. Heeger,et al.  Decoding and Reconstructing Color from Responses in Human Visual Cortex , 2009, The Journal of Neuroscience.

[44]  Aaron R. Seitz,et al.  What a difference a parameter makes: A psychophysical comparison of random dot motion algorithms , 2009, Vision Research.

[45]  Edward F. Ester,et al.  PSYCHOLOGICAL SCIENCE Research Article Stimulus-Specific Delay Activity in Human Primary Visual Cortex , 2022 .

[46]  Timothy D. Hanks,et al.  Probabilistic Population Codes for Bayesian Decision Making , 2008, Neuron.

[47]  M. Corbetta,et al.  Sensory-motor mechanisms in human parietal cortex underlie arbitrary visual decisions , 2008, Nature Neuroscience.

[48]  K. Amunts,et al.  Probabilistic maps, morphometry, and variability of cytoarchitectonic areas in the human superior parietal cortex. , 2008, Cerebral cortex.

[49]  Leslie G. Ungerleider,et al.  The neural systems that mediate human perceptual decision making , 2008, Nature Reviews Neuroscience.

[50]  S. Luck,et al.  Discrete fixed-resolution representations in visual working memory , 2008, Nature.

[51]  Brian A. Wandell,et al.  Population receptive field estimates in human visual cortex , 2008, NeuroImage.

[52]  Geoffrey M Boynton,et al.  The Representation of Behavioral Choice for Motion in Human Visual Cortex , 2007, The Journal of Neuroscience.

[53]  B. Dunn Computational Analysis , 2007 .

[54]  A. Schleicher,et al.  Ventral visual cortex in humans: Cytoarchitectonic mapping of two extrastriate areas , 2007, Human brain mapping.

[55]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[56]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[57]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[58]  R. van Ee,et al.  Visual Cortex Allows Prediction of Perceptual States during Ambiguous Structure-From-Motion , 2007, The Journal of Neuroscience.

[59]  Jean-Baptiste Poline,et al.  Inverse retinotopy: Inferring the visual content of images from brain activation patterns , 2006, NeuroImage.

[60]  Katrin Amunts,et al.  The human inferior parietal cortex: Cytoarchitectonic parcellation and interindividual variability , 2006, NeuroImage.

[61]  Leslie G. Ungerleider,et al.  Involvement of human left dorsolateral prefrontal cortex in perceptual decision making is independent of response modality , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[62]  F. Tong,et al.  Decoding Seen and Attended Motion Directions from Activity in the Human Visual Cortex , 2006, Current Biology.

[63]  Adam Kepecs,et al.  Seeing at a glance, smelling in a whiff: rapid forms of perceptual decision making , 2006, Nature Reviews Neuroscience.

[64]  A. Schleicher,et al.  Cytoarchitectonic identification and probabilistic mapping of two distinct areas within the anterior ventral bank of the human intraparietal sulcus , 2006, The Journal of comparative neurology.

[65]  Simon B. Eickhoff,et al.  A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data , 2005, NeuroImage.

[66]  Adam Gazzaley,et al.  Measuring functional connectivity during distinct stages of a cognitive task , 2004, NeuroImage.

[67]  Hauke R. Heekeren,et al.  Perceptual Decision-Making in the Human Brain , 2004 .

[68]  David J. Heeger,et al.  Neuronal correlates of perception in early visual cortex , 2003, Nature Neuroscience.

[69]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[70]  William T Newsome,et al.  Middle Temporal Visual Area Microstimulation Influences Veridical Judgments of Motion Direction , 2002, The Journal of Neuroscience.

[71]  JapkowiczNathalie,et al.  The class imbalance problem: A systematic study , 2002 .

[72]  Nathalie Japkowicz,et al.  The class imbalance problem: A systematic study , 2002, Intell. Data Anal..

[73]  O. Braddick,et al.  Brain Areas Sensitive to Coherent Visual Motion , 2001, Perception.

[74]  Karl J. Friston,et al.  A direct quantitative relationship between the functional properties of human and macaque V5 , 2000, Nature Neuroscience.

[75]  K. Amunts,et al.  Brodmann's Areas 17 and 18 Brought into Stereotaxic Space—Where and How Variable? , 2000, NeuroImage.

[76]  Wilson S. Geisler,et al.  Motion streaks provide a spatial code for motion direction , 1999, Nature.

[77]  William Prinzmetal,et al.  The Phenomenology of Attention , 1997, Consciousness and Cognition.

[78]  Anthony J. Movshon,et al.  Visual Response Properties of Striate Cortical Neurons Projecting to Area MT in Macaque Monkeys , 1996, The Journal of Neuroscience.

[79]  J. Movshon,et al.  A computational analysis of the relationship between neuronal and behavioral responses to visual motion , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[80]  W. Newsome,et al.  A selective impairment of motion perception following lesions of the middle temporal visual area (MT) , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[81]  T. Albright Direction and orientation selectivity of neurons in visual area MT of the macaque. , 1984, Journal of neurophysiology.

[82]  J J Koenderink,et al.  Detection of coherent movement in peripherally viewed random-dot patterns. , 1983, Journal of the Optical Society of America.

[83]  A. Watson,et al.  Quest: A Bayesian adaptive psychometric method , 1983, Perception & psychophysics.

[84]  B. Cohen,et al.  Quantitative analysis of the velocity characteristics of optokinetic nystagmus and optokinetic after‐nystagmus , 1977, The Journal of physiology.

[85]  R. Sekuler,et al.  Adaptation alters perceived direction of motion , 1976, Vision Research.

[86]  O. Braddick The masking of apparent motion in random-dot patterns. , 1973, Vision research.

[87]  John-Dylan Haynes,et al.  The Relationship between Perceptual Decision Variables and Confidence in the Human Brain. , 2016, Cerebral cortex.

[88]  Paul R MacNeilage,et al.  The effect of supine body position on human heading perception. , 2016, Journal of vision.

[89]  D. Steyn Large scale dynamics , 2015 .

[90]  Michael T. Tolston,et al.  Journal of Experimental Psychology : Human Perception and Performance Movement Constraints on Interpersonal Coordination and Communication , 2014 .

[91]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[92]  Sequential Sampling Models , 2022, Cognitive Choice Modeling.