Mapping brain activation and information during category-specific visual working memory.

How is working memory for different visual categories supported in the brain? Do the same principles of cortical specialization that govern the initial processing and encoding of visual stimuli also apply to their short-term maintenance? We investigated these questions with a delayed discrimination paradigm for faces, bodies, flowers, and scenes and applied both univariate and multivariate analyses to functional magnetic resonance imaging (fMRI) data. Activity during encoding followed the well-known specialization in posterior areas. During the delay interval, activity shifted to frontal and parietal regions but was not specialized for category. Conversely, activity in visual areas returned to baseline during that interval but showed some evidence of category specialization on multivariate pattern analysis (MVPA). We conclude that principles of cortical activation differ between encoding and maintenance of visual material. Whereas perceptual processes rely on specialized regions in occipitotemporal cortex, maintenance involves the activation of a frontoparietal network that seems to require little specialization at the category level. We also confirm previous findings that MVPA can extract information from fMRI signals in the absence of suprathreshold activation and that such signals from visual areas can reflect the material stored in memory.

[1]  J. Fuster Prefrontal Cortex , 2018 .

[2]  Alan D. Baddeley,et al.  Disruption of short-term memory by unattended speech : Implications for the structure of working memory , 1982 .

[3]  M. Taussig The Nervous System , 1991 .

[4]  R. Desimone,et al.  Activity of neurons in anterior inferior temporal cortex during a short- term memory task , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[5]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[6]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[7]  K. Nakamura,et al.  Mnemonic firing of neurons in the monkey temporal pole during a visual recognition memory task. , 1995, Journal of neurophysiology.

[8]  Jonathan D. Cohen,et al.  Improved Assessment of Significant Activation in Functional Magnetic Resonance Imaging (fMRI): Use of a Cluster‐Size Threshold , 1995, Magnetic resonance in medicine.

[9]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[10]  D. Heeger,et al.  Linear Systems Analysis of Functional Magnetic Resonance Imaging in Human V1 , 1996, The Journal of Neuroscience.

[11]  T. Allison,et al.  Differential Sensitivity of Human Visual Cortex to Faces, Letterstrings, and Textures: A Functional Magnetic Resonance Imaging Study , 1996, The Journal of Neuroscience.

[12]  B. Rockstroh,et al.  Contingent negative variation (CNV) and determinants of the post-imperative negative variation (PINV) in schizophrenic patients and healthy controls , 1996, Schizophrenia Research.

[13]  J. Hanley,et al.  Does articulatory suppression remove the irrelevant speech effect? , 1997, Memory.

[14]  M. D’Esposito,et al.  A Trial-Based Experimental Design for fMRI , 1997, NeuroImage.

[15]  Leslie G. Ungerleider,et al.  Transient and sustained activity in a distributed neural system for human working memory , 1997, Nature.

[16]  P. Goldman-Rakic,et al.  Areal segregation of face-processing neurons in prefrontal cortex. , 1997, Science.

[17]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[18]  Nancy Kanwisher,et al.  A cortical representation of the local visual environment , 1998, Nature.

[19]  R. Elliott,et al.  The Neural Response in Short-Term Visual Recognition Memory for Perceptual Conjunctions , 1998, NeuroImage.

[20]  Leslie G. Ungerleider,et al.  The role of prefrontal cortex in working memory: examining the contents of consciousness. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[21]  M. D’Esposito,et al.  An Area within Human Ventral Cortex Sensitive to “Building” Stimuli Evidence and Implications , 1998, Neuron.

[22]  Leslie G. Ungerleider,et al.  An area specialized for spatial working memory in human frontal cortex. , 1998, Science.

[23]  Topography of CNV and PINV in schizotypal personality. , 1998, Psychophysiology.

[24]  P S Goldman-Rakic,et al.  Face-selective neurons during passive viewing and working memory performance of rhesus monkeys: evidence for intrinsic specialization of neuronal coding. , 1999, Cerebral cortex.

[25]  N. Kanwisher,et al.  Mental Imagery of Faces and Places Activates Corresponding Stimulus-Specific Brain Regions , 2000, Journal of Cognitive Neuroscience.

[26]  N. Kanwisher,et al.  The Human Body , 2001 .

[27]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[28]  David J. Freedman,et al.  Categorical representation of visual stimuli in the primate prefrontal cortex. , 2001, Science.

[29]  Thomas Dierks,et al.  Tracking the Mind's Image in the Brain II Transcranial Magnetic Stimulation Reveals Parietal Asymmetry in Visuospatial Imagery , 2002, Neuron.

[30]  W. Singer,et al.  Distributed cortical systems in visual short-term memory revealed by event-related functional magnetic resonance imaging. , 2002, Cerebral cortex.

[31]  N. Kanwisher,et al.  How Distributed Is Visual Category Information in Human Occipito-Temporal Cortex? An fMRI Study , 2002, Neuron.

[32]  R. Savoy Functional Magnetic Resonance Imaging (fMRI) , 2002 .

[33]  Rainer Goebel,et al.  Cortical capacity constraints for visual working memory: dissociation of fMRI load effects in a fronto-parietal network , 2003, NeuroImage.

[34]  B. Postle,et al.  Seeking the Neural Substrates of Visual Working Memory Storage , 2003, Cortex.

[35]  David D. Cox,et al.  Functional magnetic resonance imaging (fMRI) “brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex , 2003, NeuroImage.

[36]  J Richard Hanley,et al.  Irrelevant speech , articulatory suppression , and phonological similarity : A test of the phonological loop model and the feature model , 2003 .

[37]  C. Curtis,et al.  Persistent activity in the prefrontal cortex during working memory , 2003, Trends in Cognitive Sciences.

[38]  Joseph B. Sala,et al.  Functional topography of a distributed neural system for spatial and nonspatial information maintenance in working memory , 2003, Neuropsychologia.

[39]  N. Kanwisher,et al.  The fusiform face area subserves face perception, not generic within-category identification , 2004, Nature Neuroscience.

[40]  Michael X. Cohen,et al.  Inferior Temporal, Prefrontal, and Hippocampal Contributions to Visual Working Memory Maintenance and Associative Memory Retrieval , 2004, The Journal of Neuroscience.

[41]  Cosimo Urgesi,et al.  Magnetic Stimulation of Extrastriate Body Area Impairs Visual Processing of Nonfacial Body Parts , 2004, Current Biology.

[42]  Andreas Nieder,et al.  The Number Domain— Can We Count on Parietal Cortex? , 2004, Neuron.

[43]  M. Goodale,et al.  An evolving view of duplex vision: separate but interacting cortical pathways for perception and action , 2004, Current Opinion in Neurobiology.

[44]  M. D’Esposito,et al.  Category-specific modulation of inferior temporal activity during working memory encoding and maintenance. , 2004, Brain research. Cognitive brain research.

[45]  Sharon L. Thompson-Schill,et al.  Learning Places from Views: Variation in Scene Processing as a Function of Experience and Navigational Ability , 2005, Journal of Cognitive Neuroscience.

[46]  P. Fries A mechanism for cognitive dynamics: neuronal communication through neuronal coherence , 2005, Trends in Cognitive Sciences.

[47]  M. D’Esposito,et al.  Directing the mind's eye: prefrontal, inferior and medial temporal mechanisms for visual working memory , 2005, Current Opinion in Neurobiology.

[48]  Conny F. Schmidt,et al.  Face perception is mediated by a distributed cortical network , 2005, Brain Research Bulletin.

[49]  David E. J. Linden,et al.  Separation of the Systems for Color and Spatial Manipulation in Working Memory Revealed by a Dual-task Procedure , 2005, Journal of Cognitive Neuroscience.

[50]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[51]  P. Downing,et al.  Within‐subject reproducibility of category‐specific visual activation with functional MRI , 2005, Human brain mapping.

[52]  Rebecca F. Schwarzlose,et al.  Separate face and body selectivity on the fusiform gyrus. , 2010, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[53]  G. Rees,et al.  Predicting the orientation of invisible stimuli from activity in human primary visual cortex , 2005, Nature Neuroscience.

[54]  Vaidehi S. Natu,et al.  Category-Specific Cortical Activity Precedes Retrieval During Memory Search , 2005, Science.

[55]  R. Goebel,et al.  Content- and Task-Specific Dissociations of Frontal Activity during Maintenance and Manipulation in Visual Working Memory , 2006, The Journal of Neuroscience.

[56]  Alison J. Wiggett,et al.  Patterns of fMRI Activity Dissociate Overlapping Functional Brain Areas that Respond to Biological Motion , 2006, Neuron.

[57]  N. Kanwisher,et al.  Domain specificity in visual cortex. , 2006, Cerebral cortex.

[58]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[59]  Rainer Goebel,et al.  Common neural substrates for visual working memory and attention , 2007, NeuroImage.

[60]  Jane E Raymond,et al.  Familiarity enhances visual working memory for faces. , 2008, Journal of experimental psychology. Human perception and performance.

[61]  David J. Freedman,et al.  Dynamic population coding of category information in inferior temporal and prefrontal cortex. , 2008, Journal of neurophysiology.

[62]  David J. Freedman,et al.  Neural mechanisms of visual categorization: Insights from neurophysiology , 2008, Neuroscience & Biobehavioral Reviews.

[63]  J. Devlin,et al.  Triple Dissociation of Faces, Bodies, and Objects in Extrastriate Cortex , 2009, Current Biology.

[64]  Edward F. Ester,et al.  PSYCHOLOGICAL SCIENCE Research Article Stimulus-Specific Delay Activity in Human Primary Visual Cortex , 2022 .

[65]  Stephen M. Smith,et al.  Threshold-free cluster enhancement: Addressing problems of smoothing, threshold dependence and localisation in cluster inference , 2009, NeuroImage.

[66]  Edward F. Ester,et al.  Spatially Global Representations in Human Primary Visual Cortex during Working Memory Maintenance , 2009, The Journal of Neuroscience.

[67]  F. Tong,et al.  Decoding reveals the contents of visual working memory in early visual areas , 2009, Nature.

[68]  Bruno B Averbeck,et al.  Rapid Sequences of Population Activity Patterns Dynamically Encode Task-Critical Spatial Information in Parietal Cortex , 2010, The Journal of Neuroscience.

[69]  P. Goldman-Rakic Circuitry of Primate Prefrontal Cortex and Regulation of Behavior by Representational Memory , 2011 .

[70]  P. Downing,et al.  Human Neuroscience , 2022 .

[71]  Paul E. Downing,et al.  A comparison of volume-based and surface-based multi-voxel pattern analysis , 2011, NeuroImage.

[72]  Helen M. Morgan,et al.  Strategic resource allocation in the human brain supports cognitive coordination of object and spatial working memory , 2010, Human brain mapping.