A matter of availability: sharper tuning for memorized than for perceived stimulus features

Abstract Our visual environment is relatively stable over time. An optimized visual system could capitalize on this by devoting less representational resources to objects that are physically present. The vividness of subjective experience, however, suggests that externally available (perceived) information is more strongly represented in neural signals than memorized information. To distinguish between these opposing predictions, we use EEG multivariate pattern analysis to quantify the representational strength of task-relevant features in anticipation of a change-detection task. Perceptual availability was manipulated between experimental blocks by either keeping the stimulus available on the screen during a 2-s delay period (perception) or removing it shortly after its initial presentation (memory). We find that task-relevant (attended) memorized features are more strongly represented than irrelevant (unattended) features. More importantly, we find that task-relevant features evoke significantly weaker representations when they are perceptually available compared with when they are unavailable. These findings demonstrate that, contrary to what subjective experience suggests, vividly perceived stimuli elicit weaker neural representations (in terms of detectable multivariate information) than the same stimuli maintained in visual working memory. We hypothesize that an efficient visual system spends little of its limited resources on the internal representation of information that is externally available anyway.

[1]  C. Paffen,et al.  Mountains of memory in a sea of uncertainty: Sampling the external world despite useful information in visual working memory , 2023, Cognition.

[2]  Anina N. Rich,et al.  The time-course of feature-based attention effects dissociated from temporal expectation and target-related processes , 2022, Scientific Reports.

[3]  P. Binda,et al.  The pupil responds spontaneously to perceived numerosity , 2021, Nature Communications.

[4]  Surya Gayet,et al.  Essential considerations for exploring visual working memory storage in the human brain , 2021, Visual Cognition.

[5]  S. van der Stigchel,et al.  Dynamic and flexible transformation and reallocation of visual working memory representations , 2021 .

[6]  J. Benda Neural adaptation , 2021, Current Biology.

[7]  A. Nobre,et al.  When Natural Behavior Engages Working Memory , 2020, Current Biology.

[8]  Peter Hagoort,et al.  Neuronal spike-rate adaptation supports working memory in language processing , 2020, Proceedings of the National Academy of Sciences.

[9]  S. van der Stigchel An embodied account of visual working memory , 2020 .

[10]  Andrea Pavan,et al.  Short- and long-term forms of neural adaptation: An ERP investigation of dynamic motion aftereffects , 2020, Cortex.

[11]  A. Belopolsky,et al.  Eye movements reveal learning and information-seeking in attentional template acquisition , 2019, Visual Cognition.

[12]  Rosyl S. Somai,et al.  Evidence for the world as an external memory: A trade-off between internal and external visual memory storage , 2020, Cortex.

[13]  J. Serences,et al.  Coexisting representations of sensory and mnemonic information in human visual cortex , 2019, Nature Neuroscience.

[14]  Jeffrey S. Johnson,et al.  The time course of encoding and maintenance of task-relevant versus irrelevant object features in working memory , 2019, Cortex.

[15]  Johannes J. Fahrenfort,et al.  High-pass filtering artifacts in multivariate classification of neural time series data , 2018, Journal of Neuroscience Methods.

[16]  Johannes Jacobus Fahrenfort,et al.  Current and future goals are represented in opposite patterns in object-selective cortex , 2018, bioRxiv.

[17]  S. Luck,et al.  Dissociable Decoding of Spatial Attention and Working Memory from EEG Oscillations and Sustained Potentials , 2018, The Journal of Neuroscience.

[18]  Anke Marit Albers,et al.  Eye Movement-Related Confounds in Neural Decoding of Visual Working Memory Representations , 2017, eNeuro.

[19]  Qing Yu,et al.  Occipital, parietal, and frontal cortices selectively maintain task-relevant features of multi-feature objects in visual working memory , 2017, NeuroImage.

[20]  Piers D L Howe,et al.  Shared processing in multiple object tracking and visual working memory in the absence of response order and task order confounds , 2017, PloS one.

[21]  Rosanne L Rademaker,et al.  How Do Visual and Parietal Cortex Contribute to Visual Short-Term Memory?12 , 2016, eNeuro.

[22]  Pierre J. Magistretti,et al.  A Cellular Perspective on Brain Energy Metabolism and Functional Imaging , 2015, Neuron.

[23]  Markus H. Sneve,et al.  Pupil size signals mental effort deployed during multiple object tracking and predicts brain activity in the dorsal attention network and the locus coeruleus. , 2014, Journal of vision.

[24]  E. Vogel,et al.  Neural Limits to Representing Objects Still within View , 2013, The Journal of Neuroscience.

[25]  T. Horowitz,et al.  Swapping or dropping? Electrophysiological measures of difficulty during multiple object tracking , 2013, Cognition.

[26]  George A Alvarez,et al.  Object-based benefits without object-based representations. , 2012, Journal of experimental psychology. General.

[27]  P. Mamassian,et al.  Predictive Properties of Visual Adaptation , 2012, Current Biology.

[28]  Edward K. Vogel,et al.  Neural Measures of Dynamic Changes in Attentive Tracking Load , 2012, Journal of Cognitive Neuroscience.

[29]  T. Carlson,et al.  High temporal resolution decoding of object position and category. , 2011, Journal of vision.

[30]  Edward F. Ester,et al.  Spatially Global Representations in Human Primary Visual Cortex during Working Memory Maintenance , 2009, The Journal of Neuroscience.

[31]  F. Tong,et al.  Decoding reveals the contents of visual working memory in early visual areas , 2009, Nature.

[32]  Edward F. Ester,et al.  PSYCHOLOGICAL SCIENCE Research Article Stimulus-Specific Delay Activity in Human Primary Visual Cortex , 2022 .

[33]  E. Vogel,et al.  Neural Measures of Individual Differences in Selecting and Tracking Multiple Moving Objects , 2008, The Journal of Neuroscience.

[34]  M. Tsodyks,et al.  Synaptic Theory of Working Memory , 2008, Science.

[35]  Valentin Dragoi,et al.  Adaptive coding of visual information in neural populations , 2008, Nature.

[36]  M. Webster,et al.  Visual adaptation: Neural, psychological and computational aspects , 2007, Vision Research.

[37]  A. Fairhall,et al.  Sensory adaptation , 2007, Current Opinion in Neurobiology.

[38]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[39]  E. Vogel,et al.  PSYCHOLOGICAL SCIENCE Research Article Visual Working Memory Represents a Fixed Number of Items Regardless of Complexity , 2022 .

[40]  Yaoda Xu Understanding the object benefit in visual short-term memory: The roles of feature proximity and connectedness , 2006, Perception & Psychophysics.

[41]  F. Tong,et al.  Decoding Seen and Attended Motion Directions from Activity in the Human Visual Cortex , 2006, Current Biology.

[42]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[43]  M. Just,et al.  Neuroindices of cognitive workload: Neuroimaging, pupillometric and event-related potential studies of brain work , 2003 .

[44]  Yaoda Xu,et al.  Encoding color and shape from different parts of an object in visual short-term memory , 2002, Perception & psychophysics.

[45]  Yuhong V. Jiang,et al.  Is visual short-term memory object based? Rejection of the “strong-object” hypothesis , 2002, Perception & psychophysics.

[46]  Yaoda Xu,et al.  Limitations of object-based feature encoding in visual short-term memory. , 2002, Journal of experimental psychology. Human perception and performance.

[47]  G. Woodman,et al.  Storage of features, conjunctions and objects in visual working memory. , 2001, Journal of experimental psychology. Human perception and performance.

[48]  Edward K. Vogel,et al.  The capacity of visual working memory for features and conjunctions , 1997, Nature.

[49]  E Marder,et al.  Memory from the dynamics of intrinsic membrane currents. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[50]  E Marder,et al.  Cellular short-term memory from a slow potassium conductance. , 1996, Journal of neurophysiology.

[51]  R W Backs,et al.  Metabolic and cardiorespiratory measures of mental effort: the effects of level of difficulty in a working memory task. , 1994, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[52]  J. O'Regan,et al.  Solving the "real" mysteries of visual perception: the world as an outside memory. , 1992, Canadian journal of psychology.

[53]  D Kahneman,et al.  Pupil Diameter and Load on Memory , 1966, Science.

[54]  M. Webster Visual Adaptation. , 2015, Annual review of vision science.

[55]  M. D’Esposito Working memory. , 2008, Handbook of clinical neurology.

[56]  D. Ballard,et al.  Memory Representations in Natural Tasks , 1995, Journal of Cognitive Neuroscience.

[57]  D. P. Andrews,et al.  Perception of contour orientation in the central fovea. I: short lines. , 1967, Vision research.