Visual Working Memory Enhances the Neural Response to Matching Visual Input

Visual working memory (VWM) is used to maintain visual information available for subsequent goal-directed behavior. The content of VWM has been shown to affect the behavioral response to concurrent visual input, suggesting that visual representations originating from VWM and from sensory input draw upon a shared neural substrate (i.e., a sensory recruitment stance on VWM storage). Here, we hypothesized that visual information maintained in VWM would enhance the neural response to concurrent visual input that matches the content of VWM. To test this hypothesis, we measured fMRI BOLD responses to task-irrelevant stimuli acquired from 15 human participants (three males) performing a concurrent delayed match-to-sample task. In this task, observers were sequentially presented with two shape stimuli and a retro-cue indicating which of the two shapes should be memorized for subsequent recognition. During the retention interval, a task-irrelevant shape (the probe) was briefly presented in the peripheral visual field, which could either match or mismatch the shape category of the memorized stimulus. We show that this probe stimulus elicited a stronger BOLD response, and allowed for increased shape-classification performance, when it matched rather than mismatched the concurrently memorized content, despite identical visual stimulation. Our results demonstrate that VWM enhances the neural response to concurrent visual input in a content-specific way. This finding is consistent with the view that neural populations involved in sensory processing are recruited for VWM storage, and it provides a common explanation for a plethora of behavioral studies in which VWM-matching visual input elicits a stronger behavioral and perceptual response. SIGNIFICANCE STATEMENT Humans heavily rely on visual information to interact with their environment and frequently must memorize such information for later use. Visual working memory allows for maintaining such visual information in the mind's eye after termination of its retinal input. It is hypothesized that information maintained in visual working memory relies on the same neural populations that process visual input. Accordingly, the content of visual working memory is known to affect our conscious perception of concurrent visual input. Here, we demonstrate for the first time that visual input elicits an enhanced neural response when it matches the content of visual working memory, both in terms of signal strength and information content.

[1]  F. Tong,et al.  Decoding reveals the contents of visual working memory in early visual areas , 2009, Nature.

[2]  Leendert van Maanen,et al.  Visual input that matches the content of visual working memory requires less (not faster) evidence sampling to reach conscious access. , 2016, Journal of vision.

[3]  S. Luck,et al.  The role of visual working memory (VWM) in the control of gaze during visual search , 2009, Attention, perception & psychophysics.

[4]  S. Edelman,et al.  Differential Processing of Objects under Various Viewing Conditions in the Human Lateral Occipital Complex , 1999, Neuron.

[5]  Zoltan Dienes,et al.  Using Bayes to get the most out of non-significant results , 2014, Front. Psychol..

[6]  Philip L. Smith,et al.  A comparison of sequential sampling models for two-choice reaction time. , 2004, Psychological review.

[7]  Edward F. Ester,et al.  Spatially Global Representations in Human Primary Visual Cortex during Working Memory Maintenance , 2009, The Journal of Neuroscience.

[8]  I. Toni,et al.  Shared Representations for Working Memory and Mental Imagery in Early Visual Cortex , 2013, Current Biology.

[9]  J. Theeuwes,et al.  Feature-based memory-driven attentional capture: visual working memory content affects visual attention. , 2006, Journal of experimental psychology. Human perception and performance.

[10]  Caspar M. Schwiedrzik,et al.  Stimulus Predictability Reduces Responses in Primary Visual Cortex , 2010, The Journal of Neuroscience.

[11]  Adam C. Riggall,et al.  The Relationship between Working Memory Storage and Elevated Activity as Measured with Functional Magnetic Resonance Imaging , 2012, The Journal of Neuroscience.

[12]  P. Roelfsema,et al.  The Distributed Nature of Working Memory , 2017, Trends in Cognitive Sciences.

[13]  C. Paffen,et al.  Cogito ergo video: Task-relevant information is involuntarily boosted into awareness. , 2015, Journal of vision.

[14]  Radoslaw Martin Cichy,et al.  Non-holistic coding of objects in lateral occipital complex with and without attention , 2015, NeuroImage.

[15]  John-Dylan Haynes,et al.  Decoding the Contents of Visual Short-Term Memory from Human Visual and Parietal Cortex , 2012, The Journal of Neuroscience.

[16]  J. Duncan,et al.  Top-Down Activation of Shape-Specific Population Codes in Visual Cortex during Mental Imagery , 2009, The Journal of Neuroscience.

[17]  K. Gegenfurtner,et al.  Visual Working Memory Contents Bias Ambiguous Structure from Motion Perception , 2013, PloS one.

[18]  Radoslaw Martin Cichy,et al.  Imagery and perception share cortical representations of content and location. , 2012, Cerebral Cortex.

[19]  Yaoda Xu,et al.  Decoding the content of visual short-term memory under distraction in occipital and parietal areas , 2015, Nature Neuroscience.

[20]  Hans Knutsson,et al.  Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates , 2016, Proceedings of the National Academy of Sciences.

[21]  G. Humphreys,et al.  Automatic guidance of attention from working memory , 2008, Trends in Cognitive Sciences.

[22]  Karl J. Friston,et al.  A Dual Role for Prediction Error in Associative Learning , 2008, Cerebral cortex.

[23]  J. Gallant,et al.  Attention to Stimulus Features Shifts Spectral Tuning of V4 Neurons during Natural Vision , 2008, Neuron.

[24]  Martin I. Sereno,et al.  Spatial maps in frontal and prefrontal cortex , 2006, NeuroImage.

[25]  Floris P. de Lange,et al.  Prior Expectations Evoke Stimulus Templates in the Primary Visual Cortex , 2014, Journal of Cognitive Neuroscience.

[26]  Stefan Van der Stigchel,et al.  Information Matching the Content of Visual Working Memory Is Prioritized for Conscious Access , 2013, Psychological science.

[27]  Edward F. Ester,et al.  PSYCHOLOGICAL SCIENCE Research Article Stimulus-Specific Delay Activity in Human Primary Visual Cortex , 2022 .

[28]  N. Tzourio-Mazoyer,et al.  Automated Anatomical Labeling of Activations in SPM Using a Macroscopic Anatomical Parcellation of the MNI MRI Single-Subject Brain , 2002, NeuroImage.

[29]  Janneke F. M. Jehee,et al.  Less Is More: Expectation Sharpens Representations in the Primary Visual Cortex , 2012, Neuron.

[30]  Karl J. Friston,et al.  Assessing the significance of focal activations using their spatial extent , 1994, Human brain mapping.

[31]  Raymond J. Dolan,et al.  fMRI Activity Patterns in Human LOC Carry Information about Object Exemplars within Category , 2008, Journal of Cognitive Neuroscience.

[32]  A. Hollingworth,et al.  Dynamic interactions between visual working memory and saccade target selection. , 2014, Journal of vision.

[33]  I. Sligte,et al.  Decoding the Contents of Visual Working Memory: Evidence for Process-Based and Content-Based Working Memory Areas? , 2013, The Journal of Neuroscience.

[34]  David Soto,et al.  Working memory biasing of visual perception without awareness , 2013, Attention, perception & psychophysics.

[35]  Sang Wook Hong,et al.  Radial bias for orientation and direction of motion modulates access to visual awareness during continuous flash suppression. , 2015, Journal of vision.

[36]  Irida Mance,et al.  Visual working memory. , 2013, Wiley interdisciplinary reviews. Cognitive science.

[37]  G. Humphreys,et al.  Early, involuntary top-down guidance of attention from working memory. , 2005, Journal of experimental psychology. Human perception and performance.

[38]  S. Dumoulin,et al.  The Relationship between Cortical Magnification Factor and Population Receptive Field Size in Human Visual Cortex: Constancies in Cortical Architecture , 2011, The Journal of Neuroscience.

[39]  Clayton E Curtis,et al.  Prioritized Maps of Space in Human Frontoparietal Cortex , 2012, The Journal of Neuroscience.

[40]  Leonardo Chelazzi,et al.  Neural basis of visual selective attention. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[41]  Edward F. Ester,et al.  Parietal and Frontal Cortex Encode Stimulus-Specific Mnemonic Representations during Visual Working Memory , 2015, Neuron.

[42]  W. K. Simmons,et al.  Circular analysis in systems neuroscience: the dangers of double dipping , 2009, Nature Neuroscience.

[43]  Martin N. Hebart,et al.  The Decoding Toolbox (TDT): a versatile software package for multivariate analyses of functional imaging data , 2015, Front. Neuroinform..

[44]  J. Reynolds,et al.  Attentional modulation of visual processing. , 2004, Annual review of neuroscience.

[45]  Yuhong Jiang,et al.  Visual working memory for simple and complex features: An fMRI study , 2006, NeuroImage.

[46]  Juha Silvanto,et al.  Transcranial magnetic stimulation reveals the content of visual short-term memory in the visual cortex , 2010, NeuroImage.

[47]  Sabine Kastner,et al.  Topographic maps in human frontal cortex revealed in memory-guided saccade and spatial working-memory tasks. , 2007, Journal of neurophysiology.

[48]  Andrew Hollingworth,et al.  Visual Working Memory Modulates Rapid Eye Movements to Simple Onset Targets , 2013, Psychological science.

[49]  J. Silvis,et al.  How memory mechanisms are a key component in the guidance of our eye movements: Evidence from the global effect , 2013, Psychonomic Bulletin & Review.

[50]  M. Chun,et al.  Dissociable neural mechanisms supporting visual short-term memory for objects , 2006, Nature.

[51]  C. Paffen,et al.  Competitive interactions in visual working memory drive access to awareness , 2017, Cortex.