Neural Correlates of Fixation Duration during Real-world Scene Viewing: Evidence from Fixation-related (FIRE) fMRI

During active scene perception, our eyes move from one location to another via saccadic eye movements, with the eyes fixating objects and scene elements for varying amounts of time. Much of the variability in fixation duration is accounted for by attentional, perceptual, and cognitive processes associated with scene analysis and comprehension. For this reason, current theories of active scene viewing attempt to account for the influence of attention and cognition on fixation duration. Yet almost nothing is known about the neurocognitive systems associated with variation in fixation duration during scene viewing. We addressed this topic using fixation-related fMRI, which involves coregistering high-resolution eye tracking and magnetic resonance scanning to conduct event-related fMRI analysis based on characteristics of eye movements. We observed that activation in visual and prefrontal executive control areas was positively correlated with fixation duration, whereas activation in ventral areas associated with scene encoding and medial superior frontal and paracentral regions associated with changing action plans was negatively correlated with fixation duration. The results suggest that fixation duration in scene viewing is controlled by cognitive processes associated with real-time scene analysis interacting with motor planning, consistent with current computational models of active vision for scene perception.

[1]  R. E. Morrison,et al.  Manipulation of stimulus onset delay in reading: evidence for parallel programming of saccades. , 1984, Journal of experimental psychology. Human perception and performance.

[2]  Erik D. Reichle,et al.  The E-Z Reader model of eye-movement control in reading: Comparisons to other models , 2003, Behavioral and Brain Sciences.

[3]  K. Rayner.  The 35th Sir Frederick Bartlett Lecture: Eye movements and attention in reading, scene perception, and visual search , 2009, Quarterly journal of experimental psychology.

[4]  Graham L. Pierce,et al.  Eye movements during scene viewing: Evidence for mixed control of fixation durations , 2008, Psychonomic bulletin & review.

[5]  J. Henderson,et al.  How are eye fixation durations controlled during scene viewing? Further evidence from a scene onset delay paradigm , 2009 .

[6]  J. Henderson,et al.  High-level scene perception. , 1999, Annual review of psychology.

[7]  J. Henderson,et al.  Accurate visual memory for previously attended objects in natural scenes , 2002 .

[8]  Andy C. H. Lee,et al.  Going beyond LTM in the MTL: A synthesis of neuropsychological and neuroimaging findings on the role of the medial temporal lobe in memory and perception , 2010, Neuropsychologia.

[9]  Steven G. Luke,et al.  Eye movement control in scene viewing and reading: evidence from the stimulus onset delay paradigm. , 2013, Journal of experimental psychology. Human perception and performance.

[10]  E. Maguire,et al.  The Human Hippocampus and Spatial and Episodic Memory , 2002, Neuron.

[11]  Steven G. Luke,et al.  The Influence of Content Meaningfulness on Eye Movements across Tasks: Evidence from Scene Viewing and Reading , 2016, Front. Psychol..

[12]  R. Passingham.  The hippocampus as a cognitive map J. O'Keefe & L. Nadel, Oxford University Press, Oxford (1978). 570 pp., £25.00 , 1979, Neuroscience.

[13]  J. Henderson,et al.  The effects of semantic consistency on eye movements during complex scene viewing , 1999 .

[14]  John M. Henderson,et al.  Cortical activation to indoor versus outdoor scenes: an fMRI study , 2007, Experimental Brain Research.

[15]  John M. Henderson.  Eye movements and scene perception , 2011 .

[16]  Frans W Cornelissen,et al.  Fixation based event‐related fmri analysis: Using eye fixations as events in functional magnetic resonance imaging to reveal cortical processing during the free exploration of visual images , 2012, Human Brain Mapping.

[17]  Carrick C. Williams,et al.  Incidental visual memory for targets and distractors in visual search , 2005, Perception & psychophysics.

[18]  Michael L. Mack,et al.  Viewing task influences eye movement control during active scene perception. , 2009, Journal of vision.

[19]  David C. Van Essen,et al.  Application of Information Technology: An Integrated Software Suite for Surface-based Analyses of Cerebral Cortex , 2001, J. Am. Medical Informatics Assoc..

[21]  Michael D. Dodd,et al.  Examining the influence of task set on eye movements and fixations. , 2011, Journal of Vision.

[22]  C. Pierrot-Deseilligny,et al.  The Role of the Human Dorsolateral Prefrontal Cortex in Ocular Motor Behavior , 2005, Annals of the New York Academy of Sciences.

[23]  John M. Henderson,et al.  Language structure in the brain: A fixation-related fMRI study of syntactic surprisal in reading , 2016, NeuroImage.

[24]  G L Shulman,et al.  INAUGURAL ARTICLE by a Recently Elected Academy Member:A default mode of brain function , 2001 .

[25]  Andy C. H. Lee,et al.  The hippocampus and visual perception , 2012, Front. Hum. Neurosci..

[26]  W. Becker,et al.  An analysis of the saccadic system by means of double step stimuli , 1979, Vision Research.

[27]  Eyal M. Reingold,et al.  Direct control of fixation times in scene viewing: Evidence from analysis of the distribution of first fixation duration , 2012 .

[28]  D. Reisberg.  The Oxford Handbook of Cognitive Psychology , 2013 .

[29]  Erik D. Reichle,et al.  Toward a model of eye movement control in reading. , 1998, Psychological review.

[30]  L. Nadel,et al.  The Hippocampus as a Cognitive Map , 1978 .

[31]  John M. Henderson,et al.  Morphology of Primary Visual Cortex Predicts Individual Differences in Fixation Duration during Text Reading , 2014, Journal of Cognitive Neuroscience.

[32]  Michael S. Beauchamp,et al.  A new method for improving functional-to-structural MRI alignment using local Pearson correlation , 2009, NeuroImage.

[33]  Nancy Millette,et al.  How People Look at Pictures , 1935 .

[34]  S. Heim,et al.  Identifying brain systems for gaze orienting during reading: fMRI investigation of the Landolt paradigm , 2013, Front. Hum. Neurosci..

[35]  J. Henderson,et al.  Neural correlates of active vision: An fMRI comparison of natural reading and scene viewing , 2015, Neuropsychologia.

[36]  Keith Rayner,et al.  Spatial frequency filtering and the direct control of fixation durations during scene viewing , 2013, Attention, perception & psychophysics.

[37]  Antje Nuthmann,et al.  Asymmetrical control of fixation durations in scene viewing , 2014, Vision Research.

[38]  J. Henderson,et al.  CRISP: a computational model of fixation durations in scene viewing. , 2010, Psychological review.

[39]  G. Loftus.  Picture perception: effects of luminance on available information and information-extraction rate. , 1985, Journal of experimental psychology. General.

[40]  D. Schacter,et al.  The Brain's Default Network , 2008, Annals of the New York Academy of Sciences.

[41]  John M. Henderson,et al.  Eye movement control during scene viewing: Immediate degradation and enhancement effects of spatial frequency filtering , 2014 .

[42]  Anna C Nobre,et al.  Subsecond Changes in Top–Down Control Exerted by Human Medial Frontal Cortex during Conflict and Action Selection: A Combined Transcranial Magnetic Stimulation–Electroencephalography Study , 2007, The Journal of Neuroscience.

[43]  D. Munoz,et al.  Look away: the anti-saccade task and the voluntary control of eye movement , 2004, Nature Reviews Neuroscience.

[44]  M. Kronbichler,et al.  Fixation-Related fMRI Analysis in the Domain of Reading Research: Using Self-Paced Eye Movements as Markers for Hemodynamic Brain Responses During Visual Letter String Processing , 2013, Cerebral cortex.

[45]  Antje Nuthmann,et al.  Eye movement control during scene viewing: immediate effects of scene luminance on fixation durations. , 2013, Journal of experimental psychology. Human perception and performance.

[46]  G H Glover,et al.  Separate neural bases of two fundamental memory processes in the human medial temporal lobe. , 1997, Science.

[47]  A R McIntosh,et al.  Neural correlates of the episodic encoding of pictures and words. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[48]  Timothy E. J. Behrens,et al.  Functional organization of the medial frontal cortex , 2007, Current Opinion in Neurobiology.

[49]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[50]  John M. Henderson,et al.  Neural correlates of fixation duration in natural reading: Evidence from fixation-related fMRI , 2015, NeuroImage.

[51]  Rodrigo M. Braga,et al.  Eye Movements during Auditory Attention Predict Individual Differences in Dorsal Attention Network Activity , 2016, Front. Hum. Neurosci..

[52]  M. Hayhoe,et al.  In what ways do eye movements contribute to everyday activities? , 2001, Vision Research.

[53]  Scott T. Grafton,et al.  Wandering Minds: The Default Network and Stimulus-Independent Thought , 2007, Science.

[54]  Nancy Kanwisher,et al.  A cortical representation of the local visual environment , 1998, Nature.

[55]  Erik D. Reichle,et al.  Eye movements in reading and information processing: Keith Rayner’s 40 year legacy , 2016 .

[56]  Abraham Z. Snyder,et al.  A default mode of brain function: A brief history of an evolving idea , 2007, NeuroImage.

[57]  B. Staresina,et al.  Perirhinal and Parahippocampal Cortices Differentially Contribute to Later Recollection of Object- and Scene-Related Event Details , 2011, The Journal of Neuroscience.

[58]  C. Pierrot-Deseilligny,et al.  Eye movement control by the cerebral cortex , 2004, Current opinion in neurology.

[59]  Andrew P. Yonelinas,et al.  Detecting Changes in Scenes: The Hippocampus Is Critical for Strength-Based Perception , 2013, Neuron.

[60]  R W Cox,et al.  Real‐time 3D image registration for functional MRI , 1999, Magnetic resonance in medicine.

[61]  Alexander Pollatsek,et al.  E–Z Reader: A cognitive-control, serial-attention model of eye-movement behavior during reading , 2006, Cognitive Systems Research.