If Visual Saliency Predicts Search, Then Why? Evidence from Normal and Gaze-Contingent Search Tasks in Natural Scenes

The Itti and Koch (Vision Research 40: 1489–1506, 2000) saliency map model has inspired a wealth of research testing the claim that bottom-up saliency determines the placement of eye fixations in natural scenes. Although saliency seems to correlate with (although not necessarily cause) fixation in free-viewing or encoding tasks, it has been suggested that visual saliency can be overridden in a search task, with saccades being planned on the basis of target features, rather than being captured by saliency. Here, we find that target regions of a scene that are salient according to this model are found quicker than control regions (Experiment 1). However, this does not seem to be altered by filtering features in the periphery using a gaze-contingent display (Experiment 2), and a deeper analysis of the eye movements made suggests that the saliency effect is instead due to the meaning of the scene regions. Experiment 3 supports this interpretation, showing that scene inversion reduces the saliency effect. These results suggest that saliency effects on search may have nothing to do with bottom-up saccade guidance.

[1]  Geoffrey M. Underwood,et al.  Cognitive Processes in Eye Guidance: Algorithms for Attention in Image Processing , 2009, Cognitive Computation.

[2]  Michael L. Mack,et al.  VISUAL SALIENCY DOES NOT ACCOUNT FOR EYE MOVEMENTS DURING VISUAL SEARCH IN REAL-WORLD SCENES , 2007 .

[3]  T. Foulsham,et al.  Eye movements during scene inspection: A test of the saliency map hypothesis , 2006 .

[4]  Jesse S. Husk,et al.  Inverting houses and textures: Investigating the characteristics of learned inversion effects , 2007, Vision Research.

[5]  G. d'Ydewalle,et al.  Early peripheral and foveal processing in fixations during scene perception , 2003 .

[6]  K. Shapiro,et al.  The contingent negative variation (CNV) event-related potential (ERP) predicts the attentional blink , 2008 .

[7]  L. Itti.  Author address: , 1999 .

[8]  Jeffrey S. Perry,et al.  Visual search: the role of peripheral information measured using gaze-contingent displays. , 2006, Journal of vision.

[9]  N. Mackworth,et al.  Cognitive determinants of fixation location during picture viewing. , 1978, Journal of experimental psychology. Human perception and performance.

[10]  Lester C. Loschky,et al.  Investigating spatial vision and dynamic attentional selection using a gaze-contingent multiresolutional display. , 2002, Journal of experimental psychology. Applied.

[11]  A. L. I︠A︡rbus.  Eye Movements and Vision , 1967 .

[12]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[13]  R. C. Langford.  How People Look at Pictures, A Study of the Psychology of Perception in Art. , 1936 .

[14]  Robin L. Hill,et al.  Eye movements : a window on mind and brain , 2007 .

[15]  Asha Iyer,et al.  Components of bottom-up gaze allocation in natural images , 2005, Vision Research.

[16]  Vassilis Cutsuridis,et al.  A Cognitive Model of Saliency, Attention, and Picture Scanning , 2009, Cognitive Computation.

[17]  P Reinagel,et al.  Natural scene statistics at the centre of gaze. , 1999, Network.

[18]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[19]  J. Henderson.  Eye movement control during visual object processing: effects of initial fixation position and semantic constraint. , 1993, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[20]  T. Foulsham,et al.  Saliency and scan patterns in the inspection of real-world scenes: Eye movements during encoding and recognition , 2009 .

[21]  Karl R. Gegenfurtner,et al.  The contribution of low-level features at the centre of gaze to saccade target selection , 2009, Vision Research.

[22]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[23]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[24]  Derrick J. Parkhurst,et al.  Modeling the role of salience in the allocation of overt visual attention , 2002, Vision Research.

[25]  J. Henderson,et al.  The effects of semantic consistency on eye movements during complex scene viewing , 1999 .

[26]  George L. Malcolm,et al.  Searching in the dark: Cognitive relevance drives attention in real-world scenes , 2009, Psychonomic bulletin & review.

[27]  L. Itti,et al.  Modeling the influence of task on attention , 2005, Vision Research.

[28]  G. McConkie,et al.  The span of the effective stimulus during a fixation in reading , 1975 .

[29]  K. Fujii,et al.  Visualization for the analysis of fluid motion , 2005, J. Vis..

[30]  J. Henderson,et al.  Initial scene representations facilitate eye movement guidance in visual search. , 2007, Journal of experimental psychology. Human perception and performance.

[31]  K. Rayner.  Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[32]  T. Foulsham,et al.  How Does the Purpose of Inspection Influence the Potency of Visual Salience in Scene Perception? , 2007, Perception.

[33]  T. A. Kelley,et al.  Effects of scene inversion on change detection of targets matched for visual salience. , 2003, Journal of vision.

[34]  D. S. Wooding,et al.  The relationship between the locations of spatial features and those of fixations made during visual examination of briefly presented images. , 1996, Spatial vision.

[35]  T. Foulsham,et al.  Quarterly Journal of Experimental Psychology: in press Visual saliency and semantic incongruency influence eye movements when , 2022 .

[36]  Tom Foulsham,et al.  Saccade control in natural images is shaped by the information visible at fixation: evidence from asymmetric gaze-contingent windows , 2011, Attention, perception & psychophysics.

[37]  Iain D. Gilchrist,et al.  Visual correlates of fixation selection: effects of scale and time , 2005, Vision Research.

[38]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[39]  M. Tarr,et al.  Becoming a “Greeble” Expert: Exploring Mechanisms for Face Recognition , 1997, Vision Research.

[40]  T. Foulsham,et al.  Short article: Does conspicuity enhance distraction? Saliency and eye landing position when searching for objects , 2009, Quarterly journal of experimental psychology.

[41]  T. Foulsham,et al.  What can saliency models predict about eye movements? Spatial and sequential aspects of fixations during encoding and recognition. , 2008, Journal of vision.

[42]  T. Valentine.  Upside-down faces: a review of the effect of inversion upon face recognition. , 1988, British journal of psychology.

[43]  Xin Chen,et al.  Real-world visual search is dominated by top-down guidance , 2006, Vision Research.

[44]  Marina Bloj,et al.  Real and predicted influence of image manipulations on eye movements during scene recognition. , 2010, Journal of vision.