Neuronal Activity in the Visual Cortex Reveals the Temporal Order of Cognitive Operations

Most mental processes consist of a number of processing steps that are executed sequentially. The timing of the individual mental operations can usually only be estimated indirectly, from the pattern of reaction times. In vision, however, many processing steps are associated with the modulation of neuronal activity in early visual areas. Here we exploited this association to elucidate the time course of neuronal activity related to each of the self-paced mental processing steps in complex visual tasks. We trained monkeys to perform two tasks, search–trace and trace–search, which required performing a sequence of two operations: a visual search for a specific color and the mental tracing of a curve. We used multielectrode recording techniques to monitor the representations of multiple visual items in area V1 at the same time and found that the relevant curve as well as the target of visual search evoked enhanced neuronal activity with a timing that depended on the order of operations. This modulation of neuronal activity in early visual areas could allow these areas to (1) act as a cognitive blackboard that permits the exchange of information between successive processing steps of a sequential visual task and to (2) contribute to the orderly progression of task-dependent endogenous attention shifts that are driven by task structure and evolve over hundreds of milliseconds.

[1]  Pieter R. Roelfsema,et al.  Object-based attention in the primary visual cortex of the macaque monkey , 1998, Nature.

[2]  B. C. Motter,et al.  Neural correlates of feature selective memory and pop-out in extrastriate area V4 , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[3]  Pieter R Roelfsema,et al.  Separable Codes for Attention and Luminance Contrast in the Primary Visual Cortex , 2010, The Journal of Neuroscience.

[4]  P. Roelfsema.  Elemental operations in vision , 2005, Trends in Cognitive Sciences.

[5]  A. Cowey,et al.  Tickling the brain: studying visual sensation, perception and cognition by transcranial magnetic stimulation. , 2001, Progress in brain research.

[6]  Pieter R Roelfsema,et al.  Subtask sequencing in the primary visual cortex , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[7]  Marty G. Woldorff,et al.  Mechanisms of Moving the Mind's Eye: Planning and Execution of Spatial Shifts of Attention , 2004, Journal of Cognitive Neuroscience.

[8]  D Mumford,et al.  On the computational architecture of the neocortex. II. The role of cortico-cortical loops. , 1992, Biological cybernetics.

[9]  Richard Reviewer-Granger.  Unified Theories of Cognition , 1991, Journal of Cognitive Neuroscience.

[10]  R. Desimone,et al.  Neural mechanisms of spatial selective attention in areas V1, V2, and V4 of macaque visual cortex. , 1997, Journal of neurophysiology.

[11]  A Pooresmaeili,et al.  Time course of attentional modulation in the frontal eye field during curve tracing. , 2009, Journal of neurophysiology.

[12]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[13]  F. Velde,et al.  From Knowing What to Knowing Where: Modeling Object-Based Attention with Feedback Disinhibition of Activation , 2001 .

[14]  Edward F. Ester,et al.  PSYCHOLOGICAL SCIENCE Research Article Stimulus-Specific Delay Activity in Human Primary Visual Cortex , 2022 .

[15]  Pieter R. Roelfsema,et al.  Noise Correlations Have Little Influence on the Coding of Selective Attention in Area V1 , 2008, Cerebral cortex.

[16]  D. Mumford.  On the computational architecture of the neocortex , 2004, Biological Cybernetics.

[17]  S. Ullman.  Visual routines , 1984, Cognition.

[18]  J. Duncan,et al.  Filtering of neural signals by focused attention in the monkey prefrontal cortex , 2002, Nature Neuroscience.

[19]  Ken Nakayama,et al.  Brightness perception and filling-in , 1991, Vision Research.

[20]  Stefan Treue,et al.  Temporal dynamics of neuronal modulation during exogenous and endogenous shifts of visual attention in macaque area MT , 2008, Proceedings of the National Academy of Sciences.

[21]  M. Paradiso,et al.  V1 response timing and surface filling-in. , 2008, Journal of neurophysiology.

[22]  Robert Desimone,et al.  Parallel and Serial Neural Mechanisms for Visual Search in Macaque Area V4 , 2005, Science.

[23]  Victor A. F. Lamme,et al.  The implementation of visual routines , 2000, Vision Research.

[24]  F. Hamker.  The reentry hypothesis: the putative interaction of the frontal eye field, ventrolateral prefrontal cortex, and areas V4, IT for attention and eye movement. , 2005, Cerebral cortex.

[25]  Chi-Hung Juan,et al.  Feedback to V1: a reverse hierarchy in vision , 2003, Experimental Brain Research.

[26]  P. Roelfsema.  Cortical algorithms for perceptual grouping. , 2006, Annual review of neuroscience.

[27]  K. Cave,et al.  Spatial Attention in Visual Search for Features and Feature Conjunctions , 1995 .

[28]  Tomaso Poggio,et al.  Fast Readout of Object Identity from Macaque Inferior Temporal Cortex , 2005, Science.

[29]  J. Assad,et al.  Neural Activity in the Middle Temporal Area and Lateral Intraparietal Area during Endogenously Cued Shifts of Attention , 2009, The Journal of Neuroscience.

[30]  C. Gilbert,et al.  Contour Saliency in Primary Visual Cortex , 2006, Neuron.

[31]  Rajesh P. N. Rao,et al.  Embodiment is the foundation, not a level , 1996, Behavioral and Brain Sciences.

[32]  P. Roelfsema,et al.  Chronic multiunit recordings in behaving animals: advantages and limitations. , 2005, Progress in brain research.

[33]  H. Spekreijse,et al.  Two distinct modes of sensory processing observed in monkey primary visual cortex (V1) , 2001, Nature Neuroscience.

[34]  V. Lamme,et al.  The distinct modes of vision offered by feedforward and recurrent processing , 2000, Trends in Neurosciences.

[35]  Pieter R. Roelfsema,et al.  The Representation of Erroneously Perceived Stimuli in the Primary Visual Cortex , 2001, Neuron.

[36]  F. van der Velde,et al.  From Knowing What to Knowing Where: Modeling Object-Based Attention with Feedback Disinhibition of Activation , 2001, Journal of Cognitive Neuroscience.

[37]  R. Desimone,et al.  Responses of Neurons in Inferior Temporal Cortex during Memory- Guided Visual Search , 1998 .

[38]  Alan Cowey,et al.  Chapter 26 Tickling the brain: studying visual sensation, perception and cognition by transcranial magnetic stimulation , 2001 .

[39]  John H. R. Maunsell,et al.  Visual response latencies in striate cortex of the macaque monkey. , 1992, Journal of neurophysiology.

[40]  David L. Sheinberg,et al.  Noticing Familiar Objects in Real World Scenes: The Role of Temporal Cortical Neurons in Natural Vision , 2001, The Journal of Neuroscience.

[41]  Takeo Watanabe,et al.  The primary visual cortex fills in color , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[42]  J Duncan,et al.  Responses of neurons in macaque area V4 during memory-guided visual search. , 2001, Cerebral cortex.

[43]  L. Bour,et al.  The Double Magnetic Induction Method for Measuring Eye Movement - Results in Monkey and Man , 1984, IEEE Transactions on Biomedical Engineering.

[44]  N. P. Bichot,et al.  Perceptual and motor processing stages identified in the activity of macaque frontal eye field neurons during visual search. , 1996, Journal of neurophysiology.

[45]  H. Spekreijse,et al.  A gradual spread of attention during mental curve tracing. , 2003, Perception & psychophysics.

[46]  P. Lennie.  The physiological basis of variations in visual latency , 1981, Vision Research.

[47]  S. Grossberg,et al.  Contrast-sensitive perceptual grouping and object-based attention in the laminar circuits of primary visual cortex , 2000, Vision Research.

[48]  R Van Rullen,et al.  Face processing using one spike per neurone. , 1998, Bio Systems.

[49]  B. C. Motter.  Focal attention produces spatially selective processing in visual cortical areas V1, V2, and V4 in the presence of competing stimuli. , 1993, Journal of neurophysiology.

[50]  William H. Press,et al.  Numerical recipes in C , 2002 .

[51]  J. Maunsell,et al.  Attention improves performance primarily by reducing interneuronal correlations , 2009, Nature Neuroscience.

[52]  T R Vidyasagar,et al.  Gating of neuronal responses in macaque primary visual cortex by an attentional spotlight , 1998, Neuroreport.

[53]  Harvey A Swadlow,et al.  Task difficulty modulates the activity of specific neuronal populations in primary visual cortex , 2008, Nature Neuroscience.

[54]  H. Spekreijse,et al.  Correlates of transsaccadic integration in the primary visual cortex of the monkey. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[55]  A. Thiele,et al.  Attention alters spatial integration in macaque V1 in an eccentricity-dependent manner , 2007, Nature Neuroscience.

[56]  F. Tong,et al.  Decoding reveals the contents of visual working memory in early visual areas , 2009, Nature.

[57]  H. Spekreijse,et al.  Masking Interrupts Figure-Ground Signals in V1 , 2002, Journal of Cognitive Neuroscience.

[58]  Shimon Ullman,et al.  Structural Saliency: The Detection Of Globally Salient Structures using A Locally Connected Network , 1988, [1988 Proceedings] Second International Conference on Computer Vision.