Visual objects interact differently during encoding and memory maintenance

The storage mechanisms of working memory are the matter of an ongoing debate. The sensory recruitment hypothesis states that memory maintenance and perceptual encoding rely on the same neural substrate. This suggests that the same cortical mechanisms that shape object perception also apply to maintained memory content. We tested this prediction using the Direction Illusion, i.e., the mutual repulsion of two concurrently visible motion directions. Participants memorized the directions of two random dot patterns for later recall. In Experiments 1 and 2, we varied the temporal separation of spatially distinct stimuli to manipulate perceptual concurrency, while keeping concurrency within working memory constant. We observed mutual motion repulsion only under simultaneous stimulus presentation, but proactive repulsion and retroactive attraction under immediate stimulus succession. At inter-stimulus intervals of 0.5 and 2 s, however, proactive repulsion vanished, while the retroactive attraction remained. In Experiment 3, we presented both stimuli at the same spatial position and observed a reappearance of the repulsion effect. Our results indicate that the repulsive mechanisms that shape object perception across space fade during the transition from a perceptual representation to a consolidated memory content. This suggests differences in the underlying structure of perceptual and mnemonic representations. The persistence of local interactions, however, indicates different mechanisms of spatially global and local feature interactions.

[1]  Srimant P Tripathy,et al.  High-capacity, transient retention of direction-of-motion information for multiple moving objects. , 2010, Journal of vision.

[2]  R. Sekuler,et al.  Similarity-based distortion of visual short-term memory is due to perceptual averaging , 2014, Vision Research.

[3]  C. Paffen,et al.  Visual Working Memory Storage Recruits Sensory Processing Areas , 2017, Trends in Cognitive Sciences.

[4]  Elkan G. Akyürek,et al.  Dynamic hidden states underlying working memory guided behaviour , 2017, Nature Neuroscience.

[5]  Juha Silvanto,et al.  How Visual Short-Term Memory Maintenance Modulates Subsequent Visual Aftereffects , 2013, Psychological science.

[6]  A. Compte,et al.  Build-up of serial dependence in color working memory , 2018, Scientific Reports.

[7]  Klaus Oberauer,et al.  Neural Evidence for a Distinction between Short-term Memory and the Focus of Attention , 2012, Journal of Cognitive Neuroscience.

[8]  A. Hollingworth Scene and position specificity in visual memory for objects. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[9]  Peter Wenderoth,et al.  The different mechanisms of the motion direction illusion and aftereffect , 2007, Vision Research.

[10]  Zoltán Vidnyánszky,et al.  Retention interval affects visual short-term memory encoding. , 2010, Journal of neurophysiology.

[11]  A. Kohn,et al.  Distinct Effects of Brief and Prolonged Adaptation on Orientation Tuning in Primary Visual Cortex , 2013, The Journal of Neuroscience.

[12]  Jason M. Scimeca,et al.  Reaffirming the Sensory Recruitment Account of Working Memory , 2018, Trends in Cognitive Sciences.

[13]  Visual Memories Bypass Normalization , 2018, Psychological science.

[14]  Ehud Zohary,et al.  Turning Symbolic: The Representation of Motion Direction in Working Memory , 2016, Front. Psychol..

[15]  Viola S. Störmer,et al.  Feature-Based Attention Elicits Surround Suppression in Feature Space , 2014, Current Biology.

[16]  C. Blakemore,et al.  Lateral inhibition between orientation detectors in the cat's visual cortex , 2004, Experimental Brain Research.

[17]  Duje Tadin,et al.  Perceptual and neural consequences of rapid motion adaptation , 2011, Proceedings of the National Academy of Sciences.

[18]  Liqiang Huang,et al.  Characterizing the Nature of Visual Conscious Access: The Distinction between Features and Locations , 2010, Journal of vision.

[19]  C Y Li,et al.  Shift in speed selectivity of visual cortical neurons: a neural basis of perceived motion contrast. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[20]  Mark D'Esposito,et al.  Flexible Coding of Visual Working Memory Representations during Distraction , 2018, The Journal of Neuroscience.

[21]  N. Qian,et al.  Axis-of-motion affects direction discrimination, not speed discrimination , 1999, Vision Research.

[22]  A. Hollingworth,et al.  Dynamic interactions between visual working memory and saccade target selection. , 2014, Journal of vision.

[23]  Joonkoo Park,et al.  Serial dependence in numerosity perception , 2018, Journal of vision.

[24]  Jochen Triesch,et al.  What’s “up”? Working memory contents can bias orientation processing , 2013, Vision Research.

[25]  S. Luck,et al.  Sudden Death and Gradual Decay in Visual Working Memory , 2009, Psychological science.

[26]  William Curran,et al.  The hierarchy of directional interactions in visual motion processing , 2008, Proceedings of the Royal Society B: Biological Sciences.

[27]  Yaoda Xu Sensory Cortex Is Nonessential in Working Memory Storage , 2018, Trends in Cognitive Sciences.

[28]  Vanda Agnes Nemes,et al.  A psychophysical investigation of human visual perceptual memory : a study of the retention of colour, spatial frequency and motion visual information by human visual short term memory mechanisms , 2011 .

[29]  Peter Wenderoth,et al.  Dichoptic reduction of the direction illusion is not due to binocular rivalry , 2010, Vision Research.

[30]  David Whitaker,et al.  The retention and disruption of color information in human short-term visual memory. , 2012, Journal of vision.

[31]  V. Lollo,et al.  Temporal characteristics of iconic memory , 1977, Nature.

[32]  R. Sekuler,et al.  Adaptation alters perceived direction of motion , 1976, Vision Research.

[33]  Paul M Bays,et al.  The precision of visual working memory is set by allocation of a shared resource. , 2009, Journal of vision.

[34]  John T. Serences,et al.  Coexisting representations of sensory and mnemonic information in human visual cortex , 2019, Nature Neuroscience.

[35]  Robert Patterson,et al.  Direction-selective adaptation and simultaneous contrast induced by stereoscopic (cyclopean) motion , 1996, Vision Research.

[36]  Haluk Öğmen,et al.  Attention and non-retinotopic feature integration. , 2010, Journal of vision.

[37]  Frans A. J. Verstraten,et al.  Perceptual manifestations of fast neural plasticity: Motion priming, rapid motion aftereffect and perceptual sensitization , 2005, Vision Research.

[38]  M. Coltheart,et al.  Iconic memory and visible persistence , 1980, Perception & psychophysics.

[39]  Nicholas E. Myers,et al.  Supraliminal But Not Subliminal Distracters Bias Working Memory Recall , 2015, Journal of experimental psychology. Human perception and performance.

[40]  Stefan Czoschke,et al.  Context information supports serial dependence of multiple visual objects across memory episodes , 2019, bioRxiv.

[41]  D Zaksas,et al.  Motion information is spatially localized in a visual working-memory task. , 2001, Journal of neurophysiology.

[42]  Yaoda Xu Reevaluating the Sensory Account of Visual Working Memory Storage , 2017, Trends in Cognitive Sciences.

[43]  Edward Awh,et al.  Alpha-Band Activity Reveals Spontaneous Representations of Spatial Position in Visual Working Memory , 2017, Current Biology.

[44]  R. Sekuler,et al.  Mutual repulsion between moving visual targets. , 1979, Science.

[45]  Michele Fornaciai,et al.  Spontaneous repulsive adaptation in the absence of attractive serial dependence. , 2019, Journal of vision.

[46]  Jeffrey N. Rouder,et al.  Bayesian t tests for accepting and rejecting the null hypothesis , 2009, Psychonomic bulletin & review.

[47]  Adam C. Riggall,et al.  Within-Category Decoding of Information in Different Attentional States in Short-Term Memory , 2016, Cerebral cortex.

[48]  William Curran,et al.  Direction Repulsion Goes Global , 2003, Current Biology.

[49]  Patrick Cavanagh,et al.  Visual persistence of figures defined by relative motion , 1992, Vision Research.

[50]  P. Wenderoth,et al.  Retinotopic encoding of the direction aftereffect , 2008, Vision Research.

[51]  Daniel P. Bliss,et al.  Serial dependence is absent at the time of perception but increases in visual working memory , 2017, bioRxiv.

[52]  P. Bays,et al.  Visual Working Memory Is Independent of the Cortical Spacing Between Memoranda , 2017, The Journal of Neuroscience.

[53]  B Moulden,et al.  A Simultaneous Shift in Apparent Direction: Further Evidence for a “Distribution-Shift” Model of Direction Coding , 1980, The Quarterly journal of experimental psychology.

[54]  G. Mather The Movement Aftereffect and a Distribution-Shift Model for Coding the Direction of Visual Movement , 1980, Perception.

[55]  Randolph Blake,et al.  Visual working memory contaminates perception , 2011, Psychonomic bulletin & review.

[56]  Edgar Erdfelder,et al.  G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences , 2007, Behavior research methods.

[57]  R. Wurtz,et al.  Response to motion in extrastriate area MSTl: center-surround interactions. , 1998, Journal of neurophysiology.

[58]  H. Wilson,et al.  Motion Integration over Space: Interaction of the Center and Surround Motion* * This research was first reported at the Annual Meeting of the Association for Research in Vision and Ophthalmology, May 1994 and 1995. , 1997, Vision Research.

[59]  O. Braddick,et al.  The temporal integration and resolution of velocity signals , 1991, Vision Research.

[60]  C. Gilbert Horizontal integration and cortical dynamics , 1992, Neuron.

[61]  Yoshimichi Ejima,et al.  Apparent contrast of a sinusoidal grating in the simultaneous presence of peripheral gratings , 1985, Vision Research.

[62]  Dag Alnæs,et al.  Modulation of Activity in Human Visual Area V1 during Memory Masking , 2011, PloS one.

[63]  F. Tong,et al.  Decoding reveals the contents of visual working memory in early visual areas , 2009, Nature.

[64]  Roger B. H. Tootell,et al.  Segregation of global and local motion processing in primate middle temporal visual area , 1992, Nature.

[65]  E. Boring,et al.  The derivation of subjective scales from just noticeable differences. , 1958, Psychological review.

[66]  Edward F. Ester,et al.  Restoring Latent Visual Working Memory Representations in Human Cortex , 2016, Neuron.

[67]  V. Lollo Temporal integration in visual memory. , 1980 .

[68]  C. Blakemore,et al.  Lateral Inhibition between Orientation Detectors in the Human Visual System , 1970, Nature.

[69]  Leonardo Chelazzi,et al.  Laws of concatenated perception: Vision goes for novelty, decisions for perseverance , 2017, bioRxiv.

[70]  S. Petersen,et al.  Memory's echo: vivid remembering reactivates sensory-specific cortex. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[71]  John-Dylan Haynes,et al.  Decoding the Contents of Visual Short-Term Memory from Human Visual and Parietal Cortex , 2012, The Journal of Neuroscience.

[72]  William Curran,et al.  The direction aftereffect is driven by adaptation of local motion detectors , 2006, Vision Research.

[73]  Rasa Ruseckaite,et al.  Human scotopic dark adaptation: Comparison of recoveries of psychophysical threshold and ERG b-wave sensitivity. , 2011, Journal of vision.

[74]  Rosanne L Rademaker,et al.  The impact of interference on short-term memory for visual orientation. , 2015, Journal of experimental psychology. Human perception and performance.

[75]  S. Luck,et al.  Discrete fixed-resolution representations in visual working memory , 2008, Nature.

[76]  Edward F. Ester,et al.  PSYCHOLOGICAL SCIENCE Research Article Stimulus-Specific Delay Activity in Human Primary Visual Cortex , 2022 .

[77]  Vasudevan Lakshminarayanan,et al.  Psychophysical estimation of speed discrimination. I. Methodology. , 2005, Journal of the Optical Society of America. A, Optics, image science, and vision.

[78]  Adam C. Riggall,et al.  The Relationship between Working Memory Storage and Elevated Activity as Measured with Functional Magnetic Resonance Imaging , 2012, The Journal of Neuroscience.

[79]  T. Sejnowski,et al.  Representation of Color Stimuli in Awake Macaque Primary Visual Cortex , 2003, Neuron.

[80]  Rosilari Bellacosa Marotti,et al.  The temporal course of recovery from brief (sub-second) adaptations to spatial contrast , 2012, Vision Research.

[81]  Gerald Westhhmer,et al.  Simultaneous orientation contrast for lines in the human fovea , 1990, Vision Research.

[82]  S. Appelle Perception and discrimination as a function of stimulus orientation: the "oblique effect" in man and animals. , 1972, Psychological bulletin.

[83]  P. Dayan,et al.  Space and time in visual context , 2007, Nature Reviews Neuroscience.

[84]  B. L. Gros,et al.  Anisotropies in visual motion perception: a fresh look. , 1998, Journal of the Optical Society of America. A, Optics, image science, and vision.

[85]  Xin Huang,et al.  Distributed and Dynamic Neural Encoding of Multiple Motion Directions of Transparently Moving Stimuli in Cortical Area MT , 2015, The Journal of Neuroscience.

[86]  Peter Wenderoth,et al.  The Hierarchical Order of Processes Underlying the Direction Illusion and the Direction Aftereffect , 2012, Perception.

[87]  Thomas Wachtler,et al.  "Tilt" in color space: Hue changes induced by chromatic surrounds. , 2015, Journal of vision.

[88]  F. D. Lange,et al.  Opposite Effects of Recent History on Perception and Decision , 2017, Current Biology.

[89]  Frank Tong,et al.  Spatial specificity of working memory representations in the early visual cortex. , 2014, Journal of vision.

[90]  Nicholas E. Myers,et al.  Benefits of Flexible Prioritization in Working Memory Can Arise Without Costs , 2017, Journal of experimental psychology. Human perception and performance.

[91]  K. Nakayama,et al.  A Velocity Analogue of Brightness Contrast , 1973, Perception.

[92]  A. Hollingworth Object-position binding in visual memory for natural scenes and object arrays. , 2007, Journal of experimental psychology. Human perception and performance.

[93]  Min-Suk Kang,et al.  Retrieval-Induced Inhibition in Short-Term Memory , 2015, Psychological science.

[94]  Adam C. Riggall,et al.  Distributed Patterns of Activity in Sensory Cortex Reflect the Precision of Multiple Items Maintained in Visual Short-Term Memory , 2013, The Journal of Neuroscience.

[95]  R. Sekuler,et al.  Distortions in recall from visual memory: two classes of attractors at work. , 2010, Journal of vision.

[96]  Katsumi Watanabe,et al.  Shape-assimilation effect: retrospective distortion of visual shapes , 2013, Attention, Perception, & Psychophysics.

[97]  T. Womelsdorf,et al.  Feature-based attention influences contextual interactions during motion repulsion , 2006, Vision Research.

[98]  Paul M Bays,et al.  Evaluating and excluding swap errors in analogue tests of working memory , 2016, Scientific Reports.

[99]  Timothy F. Brady,et al.  Modeling visual working memory with the MemToolbox. , 2013, Journal of vision.

[100]  D. Whitney,et al.  Serial Dependence in the Perception of Faces , 2014, Current Biology.

[101]  Masud Husain,et al.  Rapid Forgetting Results From Competition Over Time Between Items in Visual Working Memory , 2016, Journal of experimental psychology. Learning, memory, and cognition.

[102]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[103]  S. Luck,et al.  Interactions between visual working memory representations , 2017, Attention, perception & psychophysics.

[104]  William Curran,et al.  Directional performance in motion transparency , 2002, Vision Research.

[105]  G. Orban,et al.  Shape and Spatial Distribution of Receptive Fields and Antagonistic Motion Surrounds in the Middle Temporal Area (V5) of the Macaque , 1995, The European journal of neuroscience.

[106]  P. Wenderoth,et al.  The tilt illusion as a function of the relative and absolute lengths of test and inducing lines , 1986, Perception & psychophysics.

[107]  Bahador Bahrami,et al.  Precision of working memory for visual motion sequences and transparent motion surfaces. , 2011, Journal of vision.

[108]  Wataru Teramoto,et al.  Visual percepts modify iconic memory in humans , 2018, Scientific Reports.

[109]  David Alais,et al.  Temporal Integration of Movement: The Time-Course of Motion Streaks Revealed by Masking , 2011, PloS one.

[110]  D. Whitney,et al.  Serial dependence in visual perception , 2011, Nature Neuroscience.

[111]  J. Allman,et al.  Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local-global comparisons in visual neurons. , 1985, Annual review of neuroscience.

[112]  J. Kaiser,et al.  Two types of serial dependence in visual working memory. , 2018, British journal of psychology.

[113]  D. Whitaker,et al.  Multiple spatial frequency channels in human visual perceptual memory , 2011, Vision Research.