Salient in space, salient in time: Fixation probability predicts fixation duration during natural scene viewing.

During natural scene viewing, humans typically attend and fixate selected locations for about 200-400 ms. Two variables characterize such "overt" attention: the probability of a location being fixated, and the fixation's duration. Both variables have been widely researched, but little is known about their relation. We use a two-step approach to investigate the relation between fixation probability and duration. In the first step, we use a large corpus of fixation data. We demonstrate that fixation probability (empirical salience) predicts fixation duration across different observers and tasks. Linear mixed-effects modeling shows that this relation is explained neither by joint dependencies on simple image features (luminance, contrast, edge density) nor by spatial biases (central bias). In the second step, we experimentally manipulate some of these features. We find that fixation probability from the corpus data still predicts fixation duration for this new set of experimental data. This holds even if stimuli are deprived of low-level images features, as long as higher level scene structure remains intact. Together, this shows a robust relation between fixation duration and probability, which does not depend on simple image features. Moreover, the study exemplifies the combination of empirical research on a large corpus of data with targeted experimental manipulations.

[1]  J. Henderson,et al.  Object-based attentional selection in scene viewing. , 2010, Journal of vision.

[2]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[3]  J. Antes The time course of picture viewing. , 1974, Journal of experimental psychology.

[4]  R. Baddeley,et al.  Do we look at lights? Using mixture modelling to distinguish between low- and high-level factors in natural image viewing , 2009 .

[5]  N. Logothetis,et al.  The Effect of Learning on the Function of Monkey Extrastriate Visual Cortex , 2004, PLoS biology.

[6]  Daniel P. Kennedy,et al.  Atypical Visual Saliency in Autism Spectrum Disorder Quantified through Model-Based Eye Tracking , 2015, Neuron.

[7]  R. C. Langford How People Look at Pictures, A Study of the Psychology of Perception in Art. , 1936 .

[8]  George L. Malcolm,et al.  Eye guidance during real-world scene search: The role color plays in central and peripheral vision. , 2016, Journal of vision.

[9]  D. Foster,et al.  Influence of local scene color on fixation position in visual search. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[10]  C. Koch,et al.  Task-demands can immediately reverse the effects of sensory-driven saliency in complex visual stimuli. , 2008, Journal of vision.

[11]  Roland J. Baddeley,et al.  High frequency edges (but not contrast) predict where we fixate: A Bayesian system identification analysis , 2006, Vision Research.

[12]  Michael L. Mack,et al.  VISUAL SALIENCY DOES NOT ACCOUNT FOR EYE MOVEMENTS DURING VISUAL SEARCH IN REAL-WORLD SCENES , 2007 .

[13]  P. König,et al.  The Contributions of Image Content and Behavioral Relevancy to Overt Attention , 2014, PloS one.

[14]  B. Tatler,et al.  Deriving an appropriate baseline for describing fixation behaviour , 2014, Vision Research.

[15]  D. Braun,et al.  Phase noise and the classification of natural images , 2006, Vision Research.

[16]  Erik D. Reichle,et al.  Tracking the mind during reading via eye movements: comments on Kliegl, Nuthmann, and Engbert (2006). , 2007, Journal of experimental psychology. General.

[17]  N. Mackworth,et al.  The gaze selects informative details within pictures , 1967 .

[18]  Matthias Bethge,et al.  Information-theoretic model comparison unifies saliency metrics , 2015, Proceedings of the National Academy of Sciences.

[19]  D. S. Wooding,et al.  The relationship between the locations of spatial features and those of fixations made during visual examination of briefly presented images. , 1996, Spatial vision.

[20]  Christoph Kayser,et al.  Fixations in natural scenes: Interaction of image structure and image content , 2006, Vision Research.

[21]  Laurent Itti,et al.  Interesting objects are visually salient. , 2008, Journal of vision.

[22]  P Reinagel,et al.  Natural scene statistics at the centre of gaze. , 1999, Network.

[23]  R. Hess,et al.  What causes non-monotonic tuning of fMRI response to noisy images? , 2002, Current Biology.

[24]  M. Tinker How People Look at Pictures. , 1936 .

[25]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[26]  P. König,et al.  Beyond Correlation: Do Color Features Influence Attention in Rainforest? , 2011, Front. Hum. Neurosci..

[27]  John K. Tsotsos,et al.  Saliency, attention, and visual search: an information theoretic approach. , 2009, Journal of vision.

[28]  Antje Nuthmann,et al.  Eye movement control during scene viewing: immediate effects of scene luminance on fixation durations. , 2013, Journal of experimental psychology. Human perception and performance.

[29]  C. Koch,et al.  The relation of phase noise and luminance contrast to overt attention in complex visual stimuli. , 2006, Journal of vision.

[30]  Shenmin Zhang,et al.  What do saliency models predict? , 2014, Journal of vision.

[31]  Víctor Leborán,et al.  On the relationship between optical variability, visual saliency, and eye fixations: a computational approach. , 2012, Journal of vision.

[32]  Wolfgang Einhäuser,et al.  A new approach to modeling the influence of image features on fixation selection in scenes , 2015, Annals of the New York Academy of Sciences.

[33]  Iain D. Gilchrist,et al.  Visual correlates of fixation selection: effects of scale and time , 2005, Vision Research.

[34]  Ali Borji,et al.  What stands out in a scene? A study of human explicit saliency judgment , 2013, Vision Research.

[35]  Maciej Pajak,et al.  Object-based saccadic selection during scene perception: evidence from viewing position effects. , 2013, Journal of vision.

[36]  Reinhold Kliegl,et al.  SWIFT: a dynamical model of saccade generation during reading. , 2005, Psychological review.

[37]  Shuo Wang,et al.  Predicting human gaze beyond pixels. , 2014, Journal of vision.

[38]  W. Einhäuser,et al.  Overt attention in natural scenes: Objects dominate features , 2015, Vision Research.

[39]  Reinhold Kliegl,et al.  Semantic preview benefit during reading. , 2014, Journal of experimental psychology. Learning, memory, and cognition.

[40]  Marina Bloj,et al.  Real and predicted influence of image manipulations on eye movements during scene recognition. , 2010, Journal of vision.

[41]  Reinhold Kliegl,et al.  Toward a Perceptual-Span Theory of Distributed Processing in Reading: A Reply to Rayner, Pollatsek, Drieghe, Slattery, and Reichle (2007). , 2007 .

[42]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[43]  R. Baayen,et al.  Mixed-effects modeling with crossed random effects for subjects and items , 2008 .

[44]  Antje Nuthmann,et al.  Asymmetrical control of fixation durations in scene viewing , 2014, Vision Research.

[45]  J. Henderson,et al.  CRISP: a computational model of fixation durations in scene viewing. , 2010, Psychological review.

[46]  John M. Henderson,et al.  Neural Correlates of Fixation Duration during Real-world Scene Viewing: Evidence from Fixation-related (FIRE) fMRI , 2015, Journal of Cognitive Neuroscience.

[47]  P. König,et al.  Does luminance‐contrast contribute to a saliency map for overt visual attention? , 2003, The European journal of neuroscience.

[48]  Benjamin W Tatler,et al.  The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. , 2007, Journal of vision.

[49]  Tim K Marks,et al.  SUN: A Bayesian framework for saliency using natural statistics. , 2008, Journal of vision.

[50]  Steven G. Luke,et al.  Dissociating temporal inhibition of return and saccadic momentum across multiple eye-movement tasks. , 2014, Journal of vision.

[51]  Ali Borji,et al.  Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study , 2013, IEEE Transactions on Image Processing.

[52]  A. Nuthmann Fixation durations in scene viewing: Modeling the effects of local image features, oculomotor parameters, and task , 2016, Psychonomic bulletin & review.

[53]  K. Rayner Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[54]  Aykut Erdem,et al.  Visual saliency estimation by nonlinearly integrating features using region covariances. , 2013, Journal of vision.

[55]  Tomer Carmel,et al.  Perceptual objects capture attention , 2009, Vision Research.

[56]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[57]  Derrick J. Parkhurst,et al.  Modeling the role of salience in the allocation of overt visual attention , 2002, Vision Research.

[58]  B. Moulden,et al.  The Standard Deviation of Luminance as a Metric for Contrast in Random-Dot Images , 1990, Perception.

[59]  R. Walker,et al.  A model of saccade generation based on parallel processing and competitive inhibition , 1999, Behavioral and Brain Sciences.

[60]  Wei-Song Lin,et al.  A computational visual saliency model based on statistics and machine learning. , 2014, Journal of vision.

[61]  Christopher M. Masciocchi,et al.  Everyone knows what is interesting: salient locations which should be fixated. , 2009, Journal of vision.

[62]  Antje Nuthmann,et al.  Mechanisms of saccadic decision making while encoding naturalistic scenes. , 2015, Journal of vision.

[63]  H. Schielzeth Simple means to improve the interpretability of regression coefficients , 2010 .

[64]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[65]  P. Bex,et al.  Spatial frequency, phase, and the contrast of natural images. , 2002, Journal of the Optical Society of America. A, Optics, image science, and vision.

[66]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[67]  A. Nuthmann,et al.  Time course of pseudoneglect in scene viewing , 2014, Cortex.

[68]  Keith Rayner,et al.  Do successor effects in reading reflect lexical parafoveal processing? Evidence from corpus-based and experimental eye movement data , 2015 .

[69]  Ralf Engbert,et al.  Tracking the mind during reading: the influence of past, present, and future words on fixation durations. , 2006, Journal of experimental psychology. General.

[70]  Nathalie Guyader,et al.  When viewing natural scenes, do abnormal colors impact on spatial or temporal parameters of eye movements? , 2012, Journal of vision.

[71]  J. Henderson,et al.  Using CRISP to model global characteristics of fixation durations in scene viewing and reading with a common mechanism , 2012 .

[72]  Erik D. Reichle,et al.  The E-Z Reader model of eye-movement control in reading: Comparisons to other models , 2003, Behavioral and Brain Sciences.

[73]  Peter König,et al.  Overt Attention and Context Factors: The Impact of Repeated Presentations, Image Type, and Individual Motivation , 2011, PloS one.

[74]  Nikos K. Logothetis,et al.  Nonmonotonic noise tuning of BOLD fMRI signal to natural images in the visual cortex of the anesthetized monkey , 2001, Current Biology.

[75]  D. Barr,et al.  Random effects structure for confirmatory hypothesis testing: Keep it maximal. , 2013, Journal of memory and language.

[76]  H. Ritter,et al.  Disambiguating Complex Visual Information: Towards Communication of Personal Views of a Scene , 1996, Perception.

[77]  D. Ballard,et al.  Eye guidance in natural vision: reinterpreting salience. , 2011, Journal of vision.