Visual memory capacity in transsaccadic integration

How we perceive the visual world as stable and unified suggests the existence of transsaccadic integration that retains and integrates visual information from one eye fixation to another eye fixation across saccadic eye movements. However, the capacity of transsaccadic integration is still a subject of controversy. We tested our subjects’ memory capacity of two basic visual features, i.e. luminance (Experiment 1) and orientation (Experiment 2), both within a single fixation (i.e. visual working memory) and between separate fixations (i.e. transsaccadic memory). Experiment 2 was repeated, but attention allocation was manipulated using attentional cues at either the target or distracter (Experiment 3). Subjects were able to retain 3–4 objects in transsaccadic memory for luminance and orientation; errors generally increased as saccade size increased; and, subjects were more accurate when attention was allocated to the same location as the impending target. These results were modelled by inputting a noisy extra-retinal signal into an eye-centered feature map. Our results suggest that transsaccadic memory has a similar capacity for storing simple visual features as basic visual memory, but this capacity is dependent both on the metrics of the saccade and allocation of attention.

[1]  Ronald A. Rensink,et al.  TO SEE OR NOT TO SEE: The Need for Attention to Perceive Changes in Scenes , 1997 .

[2]  P. Goldman-Rakic,et al.  Human Brain Mapping 6:14–32(1998) � Dissociation of Mnemonic and Perceptual Processes During Spatial and Nonspatial Working Memory Using fMRI , 2022 .

[3]  G. Legge,et al.  The importance of eye movements in the analysis of simple patterns , 1986, Vision Research.

[4]  D. V. van Essen,et al.  Retinotopic organization of human visual cortex mapped with positron- emission tomography , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[5]  D. Navon Forest before trees: The precedence of global features in visual perception , 1977, Cognitive Psychology.

[6]  L. Matin,et al.  Saccadic suppression of displacement: Separate influences of saccade size and of target retinal eccentricity , 1997, Vision Research.

[7]  George E Stelmach,et al.  Pointing to an allocentric and egocentric remembered target. , 2004, Motor control.

[8]  T. Sawaguchi,et al.  Monkey prefrontal neuronal activity coding the forthcoming saccade in an oculomotor delayed matching-to-sample task. , 1998, Journal of neurophysiology.

[9]  J D Crawford,et al.  Spatial transformations for eye-hand coordination. , 2004, Journal of neurophysiology.

[10]  C. Eriksen,et al.  Allocation of attention in the visual field. , 1985, Journal of experimental psychology. Human perception and performance.

[11]  B. Tatler Characterising the Visual Buffer: Real-World Evidence for Overwriting Early in Each Fixation , 2001, Perception.

[12]  G. Woodman,et al.  Storage of features, conjunctions and objects in visual working memory. , 2001, Journal of experimental psychology. Human perception and performance.

[13]  Andrew Hollingworth,et al.  Eye Movements During Scene Viewing: An Overview , 1998 .

[14]  D. Robinson,et al.  A METHOD OF MEASURING EYE MOVEMENT USING A SCLERAL SEARCH COIL IN A MAGNETIC FIELD. , 1963, IEEE transactions on bio-medical engineering.

[15]  D. LaBerge,et al.  Shifting attention in visual space: tests of moving-spotlight models versus an activity-distribution model. , 1997, Journal of experimental psychology. Human perception and performance.

[16]  Thomas Metzinger,et al.  Conference on Neural Correlates of Consciousness: Empirical and Conceptual Questions , 1998, Consciousness and Cognition.

[17]  D. Simons In Sight, Out of Mind: When Object Representations Fail , 1996 .

[18]  K. Rayner,et al.  Is visual information integrated across saccades? , 1983, Perception & psychophysics.

[19]  H. Honda The time courses of visual mislocalization and of extraretinal eye position signals at the time of vertical saccades , 1991, Vision Research.

[20]  D. E. Irwin Integrating Information Across Saccadic Eye Movements , 1996 .

[21]  N Costes,et al.  Double dissociation in neural correlates of visual working memory: a PET study. , 2005, Brain research. Cognitive brain research.

[22]  J. J. Koenderink,et al.  Change in perceived spatial directions due to context , 2000, Perception & psychophysics.

[23]  T. Vilis,et al.  Gaze-Centered Updating of Visual Space in Human Parietal Cortex , 2003, The Journal of Neuroscience.

[24]  S. Mateeff,et al.  Peripheral vision and perceived visual direction , 1983, Biological Cybernetics.

[25]  Edward K. Vogel,et al.  The capacity of visual working memory for features and conjunctions , 1997, Nature.

[26]  Iain D Gilchrist,et al.  The Time Course of Abstract Visual Representation , 2003, Perception.

[27]  Christopher B. Currie,et al.  Visual stability across saccades while viewing complex pictures. , 1995, Journal of experimental psychology. Human perception and performance.

[28]  L. Matin,et al.  The influence of saccade length on the saccadic suppression of displacement detection , 1990, Perception & psychophysics.

[29]  R A Abrams,et al.  Speed and accuracy of saccadic eye movements: characteristics of impulse variability in the oculomotor system. , 1989, Journal of experimental psychology. Human perception and performance.

[30]  Stanley Coren,et al.  Sensation and perception , 1979 .

[31]  H Spekreijse,et al.  A Neural Correlate of Working Memory in the Monkey Primary Visual Cortex , 2001, Science.

[32]  P. Cavanagh,et al.  The Capacity of Visual Short-Term Memory is Set Both by Visual Information Load and by Number of Objects , 2004, Psychological science.

[33]  C. Curtis,et al.  Persistent activity in the prefrontal cortex during working memory , 2003, Trends in Cognitive Sciences.

[34]  Dirk Kerzel,et al.  The locus of “memory displacement” is at least partially perceptual: Effects of velocity, expectation, friction, memory averaging, and weight , 2002, Perception & psychophysics.

[35]  E. DeYoe,et al.  Mapping striate and extrastriate visual areas in human cerebral cortex. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[36]  R. W. Ditchburn Eye-movements and visual perception , 1973 .

[37]  D. Robinson Eye movements evoked by collicular stimulation in the alert monkey. , 1972, Vision research.

[38]  Sven Panis,et al.  Transsaccadic integration of bystander locations , 2004 .

[39]  Heiner Deubel,et al.  Transsaccadic memory of position and form. , 2002, Progress in brain research.

[40]  Earl K. Miller,et al.  Selective representation of relevant information by neurons in the primate prefrontal cortex , 1998, Nature.

[41]  D. Sparks,et al.  The deep layers of the superior colliculus. , 1989, Reviews of oculomotor research.

[42]  Joaquín M. Fuster,et al.  Executive frontal functions , 2000, Experimental Brain Research.

[43]  Edward E. Smith,et al.  Working Memory: A View from Neuroimaging , 1997, Cognitive Psychology.

[44]  Filip Germeys,et al.  Transsaccadic perception of saccade target and flanker objects. , 2002, Journal of experimental psychology. Human perception and performance.

[45]  John K. Tsotsos,et al.  The selective tuning model of attention: psychophysical evidence for a suppressive annulus around an attended item , 2003, Vision Research.

[46]  Leslie G. Ungerleider,et al.  An area specialized for spatial working memory in human frontal cortex. , 1998, Science.

[47]  J R Duhamel,et al.  The updating of the representation of visual space in parietal cortex by intended eye movements. , 1992, Science.

[48]  P. Goldman-Rakic,et al.  Dissociation of object and spatial processing domains in primate prefrontal cortex. , 1993, Science.

[49]  M. Goldberg,et al.  Spatial processing in the monkey frontal eye field. I. Predictive visual responses. , 1997, Journal of neurophysiology.

[50]  H. Honda Saccade-contingent displacement of the apparent position of visual stimuli flashed on a dimly illuminated structured background , 1993, Vision Research.

[51]  O'Regan Jk,et al.  Integrating visual information from successive fixations: does trans-saccadic fusion exist? , 1983 .

[52]  W Pieter Medendorp,et al.  Rotational Remapping in Human Spatial Memory during Eye and Head Motion , 2002, The Journal of Neuroscience.

[53]  D. E. Irwin Memory for position and identity across eye movements. , 1992 .

[54]  J. Grimes On the failure to detect changes in scenes across saccades. , 1996 .

[55]  J. Crawford,et al.  Transsaccadic integration of visual features in a line intersection task , 2006, Experimental Brain Research.

[56]  P. Goldman-Rakic,et al.  Dorsolateral prefrontal lesions and oculomotor delayed-response performance: evidence for mnemonic "scotomas" , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[57]  Leslie G. Ungerleider,et al.  The role of prefrontal cortex in working memory: examining the contents of consciousness. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[58]  James T. Enns,et al.  Change Detection: Paying Attention To Detail , 2000 .

[59]  J P Thomas,et al.  Bandwidths of orientation channels in human vision. , 1979, Journal of the Optical Society of America.

[60]  Heiner Deubel,et al.  Relative mislocalization of briefly presented stimuli in the retinal periphery , 1999, Perception & psychophysics.

[61]  G. McConkie,et al.  Integrating information across eye movements , 1980, Cognitive Psychology.

[62]  Karl Verfaillie,et al.  Transsaccadic memory for the egocentric and allocentric position of a biological-motion walker. , 1997 .

[63]  W. E. Collins,et al.  Integrating pictorial information across eye movements. , 1984, Journal of experimental psychology. General.

[64]  Shinsuke Shimojo,et al.  Compression of space in visual memory , 2001, Vision Research.

[65]  Leslie G. Ungerleider,et al.  Object and spatial visual working memory activate separate neural systems in human cortex. , 1996, Cerebral cortex.

[66]  Colin Blakemore,et al.  Vision: Coding and Efficiency , 1991 .

[67]  David E. Irwin,et al.  The role of the saccade target object in the perception of a visually stable world , 2000, Perception & psychophysics.

[68]  B. Bridgeman,et al.  Failure to integrate visual information from successive fixations , 1983 .

[69]  Masataka Watanabe,et al.  Prefrontal unit activity and delayed response: Relation to cue location versus direction of response , 1976, Brain Research.

[70]  L. Riggs,et al.  Involuntary motions of the eye during monocular fixation. , 1950, Journal of experimental psychology.

[71]  R. Andersen,et al.  Memory related motor planning activity in posterior parietal cortex of macaque , 1988, Experimental Brain Research.

[72]  A. H. C. van der Heijden,et al.  Sources of position-perception error for small isolated targets , 1999, Psychological research.

[73]  Madeleine Schlag-Rey,et al.  Testing quasi-visual neurons in the monkey's frontal eye field with the triple-step paradigm , 1999, Experimental Brain Research.

[74]  S. Yantis,et al.  Cortical mechanisms of feature-based attentional control. , 2003, Cerebral cortex.

[75]  Mingsha Zhang,et al.  Persistent LIP activity in memory antisaccades: working memory for a sensorimotor transformation. , 2004, Journal of neurophysiology.

[76]  David Melcher,et al.  Persistence of visual memory for scenes , 2001, Nature.

[77]  J. Duncan,et al.  Visual search and stimulus similarity. , 1989, Psychological review.

[78]  David E. Irwin Robert D. Gordon Eye Movements, Attention and Trans-saccadic Memory , 1998 .

[79]  David E. Irwin,et al.  Visual Search has Memory , 2001, Psychological science.

[80]  D. Simons,et al.  CHAPTER 13 – Change Blindness , 2005 .

[81]  C. Blakemore,et al.  Orientation Specificity and Spatial Selectivity in Human Vision , 1973, Perception.

[82]  H. Deubel,et al.  Saccade target selection and object recognition: Evidence for a common attentional mechanism , 1996, Vision Research.

[83]  Bart Krekelberg,et al.  Postsaccadic visual references generate presaccadic compression of space , 2000, Nature.

[84]  David E. Irwin,et al.  Integrating visual information from successive fixations. , 1982, Science.

[85]  G. Amirkhiabani,et al.  ROLE OF ECCENTRICITY AND SIZE IN THE GLOBAL PRECEDENCE EFFECT , 1996 .

[86]  M. Hayhoe,et al.  Integration of Form across Saccadic Eye Movements , 1991, Perception.

[87]  M. Goldberg,et al.  Oculocentric spatial representation in parietal cortex. , 1995, Cerebral cortex.

[88]  K. Verfaillie,et al.  Transsaccadic Coding of Object Position: Effects of Saccadic Status and Allocentric Reference Frame , 2001 .

[89]  T L Hubbard,et al.  Cognitive representation of motion: evidence for friction and gravity analogues. , 1995, Journal of experimental psychology. Learning, memory, and cognition.

[90]  David E. Irwin Information integration across saccadic eye movements , 1991, Cognitive Psychology.

[91]  Karl Christoph Klauer,et al.  Double dissociations in visual and spatial short-term memory. , 2004, Journal of experimental psychology. General.

[92]  D. E. Irwin,et al.  How Much Memory Does Oculomotor Search Have? , 2003, Psychological science.

[93]  D. Navon,et al.  Does global precedence really depend on visual angle? , 1983, Journal of experimental psychology. Human perception and performance.

[94]  J. Henderson,et al.  The Role of Fixation Position in Detecting Scene Changes Across Saccades , 1999 .

[95]  R. Desimone,et al.  Competitive Mechanisms Subserve Attention in Macaque Areas V2 and V4 , 1999, The Journal of Neuroscience.

[96]  M. Goodale,et al.  Separate visual pathways for perception and action , 1992, Trends in Neurosciences.

[97]  Ronald A. Rensink,et al.  Picture Changes During Blinks: Looking Without Seeing and Seeing Without Looking , 2000 .

[98]  Laura A. Carlson,et al.  Transsaccadic Coding of Multiple Objects and Features , 2001 .

[99]  John K. Tsotsos,et al.  Direct neurophysiological evidence for spatial suppression surrounding the focus of attention in vision. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[100]  D. E. Irwin,et al.  Integration and accumulation of information across saccadic eye movements. , 1996 .

[101]  P. Goldman-Rakic,et al.  Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex. , 1989, Journal of neurophysiology.

[102]  M. Pickering,et al.  Eye guidance in reading and scene perception , 1998 .

[103]  G. Woodman,et al.  Voluntary and automatic attentional control of visual working memory , 2002 .

[104]  J. Douglas Crawford,et al.  Optimal transsaccadic integration explains distorted spatial perception , 2003, Nature.

[105]  J. Fuster,et al.  Inferotemporal neurons distinguish and retain behaviorally relevant features of visual stimuli. , 1981, Science.

[106]  Leslie G. Ungerleider,et al.  Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys , 1982, Behavioural Brain Research.

[107]  Bruce Bridgeman,et al.  A theory of visual stability across saccadic eye movements , 1994, Behavioral and Brain Sciences.

[108]  Harry Singer,et al.  Theoretical models and processes of reading , 1976 .

[109]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[110]  P. Goldman-Rakic,et al.  Prefrontal neuronal activity in rhesus monkeys performing a delayed anti-saccade task , 1993, Nature.

[111]  J. O'Regan,et al.  Solving the "real" mysteries of visual perception: the world as an outside memory. , 1992, Canadian journal of psychology.

[112]  Iain D Gilchrist,et al.  Visual Memory for Objects in Natural Scenes: From Fixations to Object Files , 2005, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[113]  U. Castiello,et al.  Size of the attentional focus and efficiency of processing. , 1990, Acta psychologica.

[114]  M. Posner,et al.  Attention and the detection of signals. , 1980, Journal of experimental psychology.

[115]  Donald L. Fisher,et al.  Central capacity limits in consistent mapping, visual search tasks: Four channels or more? , 1984, Cognitive Psychology.

[116]  William C. Ogden,et al.  Attended and unattended processing modes: The role of set for spatial location , 2014 .

[117]  R. Wurtz,et al.  A Pathway in Primate Brain for Internal Monitoring of Movements , 2002, Science.

[118]  G. McConkie,et al.  Is visual information integrated across successive fixations in reading? , 1979, Perception & psychophysics.

[119]  C L Colby,et al.  Visual, saccade-related, and cognitive activation of single neurons in monkey extrastriate area V3A. , 2000, Journal of neurophysiology.

[120]  Claus Bundesen,et al.  Visual Selective Attention: Outlines of a Choice Model, a Race Model and a Computational Theory , 1998 .

[121]  P H Schiller,et al.  Visual representations during saccadic eye movements. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[122]  Marcus Kaiser,et al.  Perisaccadic Mislocalization Orthogonal to Saccade Direction , 2004, Neuron.

[123]  C. Eriksen,et al.  Visual attention within and around the field of focal attention: A zoom lens model , 1986, Perception & psychophysics.

[124]  David E. Irwin,et al.  Evidence against visual integration across saccadic eye movements , 1983, Perception & psychophysics.

[125]  Shintaro Funahashi,et al.  Prefrontal task-related activity representing visual cue location or saccade direction in spatial working memory tasks. , 2002, Journal of neurophysiology.

[126]  N J Gandhi,et al.  Comparison of saccades perturbed by stimulation of the rostral superior colliculus, the caudal superior colliculus, and the omnipause neuron region. , 1999, Journal of neurophysiology.

[127]  J. Baird,et al.  Global precedence in visual pattern recognition , 1984, Perception & psychophysics.

[128]  Trichur R Vidyasagar,et al.  The role of the magnocellular pathway in serial deployment of visual attention , 2004, The European journal of neuroscience.

[129]  B. Dosher,et al.  The role of attention in the programming of saccades , 1995, Vision Research.

[130]  J. O'Regan,et al.  Integrating visual information from successive fixations:Does trans-saccadic fusion exist? , 1983, Vision Research.

[131]  W. Heide,et al.  Cortical control of double‐step saccades: Implications for spatial orientation , 1995, Annals of neurology.

[132]  D. E. Irwin,et al.  Eye movements and scene perception: Memory for things observed , 2002, Perception & psychophysics.

[133]  Peter De Graef,et al.  Transsaccadic memory for visual object detail. , 2002, Progress in brain research.

[134]  Lawrence H Snyder,et al.  Spatial memory following shifts of gaze. I. Saccades to memorized world-fixed and gaze-fixed targets. , 2003, Journal of neurophysiology.

[135]  C. Blakemore,et al.  Vision: The iconic bottleneck and the tenuous link between early visual processing and perception , 1990 .

[136]  K. Rayner Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[137]  R. Kimchi Primacy of wholistic processing and global/local paradigm: a critical review. , 1992, Psychological bulletin.