A novel measure to determine viewing priority and its neural correlates in the human brain.

A key property of human visual behavior is the very frequent movement of our eyes to potentially relevant information in the environment. Observers thus continuously have to prioritize information for directing their eyes to. Research in this field has been hampered by a lack of appropriate measures and tools. Here, we propose and validate a novel measure of priority that takes advantage of the variability in the natural viewing behavior of individual observers. In short, our measure assumes that priority is low when observers' gaze behavior is inconsistent and high when it is very consistent. We calculated priority for gaze data obtained during an experiment in which participants viewed dynamic natural scenes while we simultaneously recorded their gaze position and brain activity using functional magnetic resonance imaging. Our priority measure shows only limited correlation with various saliency, surprise, and motion measures, indicating it is assessing a distinct property of visual behavior. Finally, we correlated our priority measure with the BOLD signal, thereby revealing activity in a select number of human occipital and parietal areas. This suggests the presence of a cortical network involved in computing and representing viewing priority. We conclude that our new analysis method allows for empirically establishing the priority of events in near-natural vision paradigms.

[1]  M. Goodale,et al.  Chapter 28 Visual pathways to perception and action , 1993 .

[2]  James C. Bezdek,et al.  Models for Pattern Recognition , 1981 .

[3]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[4]  J. Downar,et al.  A multimodal cortical network for the detection of changes in the sensory environment , 2000, Nature Neuroscience.

[5]  J. Henderson.  Human gaze control during real-world scene perception , 2003, Trends in Cognitive Sciences.

[6]  Joel R. Meyer,et al.  A large-scale distributed network for covert spatial attention: further anatomical delineation based on stringent behavioural and cognitive controls. , 1999, Brain : a journal of neurology.

[7]  Remco Renken,et al.  Simultaneous EMG‐functional MRI recordings can directly relate hyperkinetic movements to brain activity , 2008, Human brain mapping.

[8]  Karl J. Friston,et al.  The choice of basis functions in event-related fMRI , 2001, NeuroImage.

[9]  M. Tinker.  How People Look at Pictures. , 1936 .

[10]  Frans W Cornelissen,et al.  Fixation based event‐related fmri analysis: Using eye fixations as events in functional magnetic resonance imaging to reveal cortical processing during the free exploration of visual images , 2012, Human Brain Mapping.

[11]  Matthew H Tong,et al.  of the Annual Meeting of the Cognitive Science Society Title SUNDAy : Saliency Using Natural Statistics for Dynamic Analysis of Scenes Permalink , 2009 .

[12]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[13]  G. Rizzolatti,et al.  Reorienting attention across the horizontal and vertical meridians: Evidence in favor of a premotor theory of attention , 1987, Neuropsychologia.

[14]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[15]  S. Zeki,et al.  Functional brain mapping during free viewing of natural scenes , 2004, Human brain mapping.

[16]  A. Cavanna,et al.  The precuneus: a review of its functional anatomy and behavioural correlates. , 2006, Brain : a journal of neurology.

[17]  Jillian H. Fecteau,et al.  Salience, relevance, and firing: a priority map for target selection , 2006, Trends in Cognitive Sciences.

[18]  D. Ballard,et al.  Eye movements in natural behavior , 2005, Trends in Cognitive Sciences.

[19]  J. Bisley,et al.  Evidence for differential top-down and bottom-up suppression in posterior parietal cortex , 2013, Philosophical Transactions of the Royal Society B: Biological Sciences.

[20]  Michael L. Mack,et al.  VISUAL SALIENCY DOES NOT ACCOUNT FOR EYE MOVEMENTS DURING VISUAL SEARCH IN REAL-WORLD SCENES , 2007 .

[21]  R. Andersen,et al.  Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[22]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[23]  David C. Van Essen,et al.  Application of Information Technology: An Integrated Software Suite for Surface-based Analyses of Cerebral Cortex , 2001, J. Am. Medical Informatics Assoc..

[24]  Michael S. Beauchamp,et al.  A Parametric fMRI Study of Overt and Covert Shifts of Visuospatial Attention , 2001, NeuroImage.

[25]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[26]  M. Goldberg,et al.  Attention, intention, and priority in the parietal lobe. , 2010, Annual review of neuroscience.

[27]  Ali Borji,et al.  What stands out in a scene? A study of human explicit saliency judgment , 2013, Vision Research.

[28]  Jean-Luc Anton,et al.  Region of interest analysis using an SPM toolbox , 2010 .

[29]  Mary M Hayhoe,et al.  Spatial memory and saccadic targeting in a natural task. , 2005, Journal of vision.

[30]  Alan C. Evans,et al.  A new anatomical landmark for reliable identification of human area V5/MT: a quantitative analysis of sulcal patterning. , 2000, Cerebral cortex.

[31]  Thomas Martinetz,et al.  Eye movement predictions on natural videos , 2006, Neurocomputing.

[32]  M. Corbetta,et al.  Right TPJ deactivation during visual search: functional significance and support for a filter hypothesis. , 2007, Cerebral cortex.

[33]  Mary M Hayhoe,et al.  Task and context determine where you look. , 2016, Journal of vision.

[34]  J. Downar,et al.  A cortical network sensitive to stimulus salience in a neutral behavioral context across multiple sensory modalities. , 2002, Journal of neurophysiology.

[35]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[36]  M. Goodale,et al.  Visual pathways to perception and action. , 1993, Progress in brain research.

[37]  Erhardt Barth,et al.  Learned saliency transformations for gaze guidance , 2011, Electronic Imaging.

[38]  Thomas Martinetz,et al.  Variability of eye movements when viewing dynamic natural scenes. , 2010, Journal of vision.

[39]  F. Cornelissen,et al.  Attentional Window Set by Expected Relevance of Environmental Signals , 2011, PloS one.

[40]  A. Kingstone,et al.  Saliency does not account for fixations to eyes within social scenes , 2009, Vision Research.

[41]  John K. Tsotsos,et al.  Saliency, attention, and visual search: an information theoretic approach. , 2009, Journal of vision.

[42]  P. C. Murphy,et al.  Cerebral Cortex , 2017, Cerebral Cortex.

[43]  Erhardt Barth.  The Minors of the Structure Tensor , 2000, DAGM-Symposium.

[44]  Carlo Alberto Marzi,et al.  FMRI correlates of visuo‐spatial reorienting investigated with an attention shifting double‐cue paradigm , 2009, Human brain mapping.

[45]  Frans W Cornelissen,et al.  The Eyelink Toolbox: Eye tracking with MATLAB and the Psychophysics Toolbox , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.