Searching in the dark: Cognitive relevance drives attention in real-world scenes

We investigated whether the deployment of attention in scenes is better explained by visual salience or by cognitive relevance. In two experiments, participants searched for target objects in scene photographs. The objects appeared in semantically appropriate locations but were not visually salient within their scenes. Search was fast and efficient, with participants much more likely to look to the targets than to the salient regions. This difference was apparent from the first fixation and held regardless of whether participants were familiar with the visual form of the search targets. In the majority of trials, salient regions were not fixated. The critical effects were observed for all 24 participants across the two experiments. We outline a cognitive relevance framework to account for the control of attention and fixation in scenes.

[1]  Nancy Millette,et al.  How People Look at Pictures , 1935 .

[2]  M. Tinker.  How People Look at Pictures. , 1936 .

[3]  N. Mackworth,et al.  The gaze selects informative details within pictures , 1967 .

[4]  A. L. I︠A︡rbus.  Eye Movements and Vision , 1967 .

[5]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[6]  J. Antes.  The time course of picture viewing. , 1974, Journal of experimental psychology.

[7]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[8]  J. Wolfe,et al.  Guided Search 2.0 A revised model of visual search , 1994, Psychonomic bulletin & review.

[9]  Andrew Hollingworth,et al.  Eye Movements During Scene Viewing: An Overview , 1998 .

[10]  M. Pickering,et al.  Eye guidance in reading and scene perception , 1998 .

[11]  J. Henderson,et al.  The effects of semantic consistency on eye movements during complex scene viewing , 1999 .

[12]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[13]  M. Hayhoe,et al.  In what ways do eye movements contribute to everyday activities? , 2001, Vision Research.

[14]  Rajesh P. N. Rao,et al.  Eye movements in iconic visual search , 2002, Vision Research.

[15]  Derrick J. Parkhurst,et al.  Modeling the role of salience in the allocation of overt visual attention , 2002, Vision Research.

[16]  J. Henderson.  Human gaze control during real-world scene perception , 2003, Trends in Cognitive Sciences.

[17]  Antonio Torralba,et al.  Modeling global scene factors in attention. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[18]  K. Turano,et al.  Oculomotor strategies for the direction of gaze tested with a real-world activity , 2003, Vision Research.

[19]  Mary M Hayhoe,et al.  Visual memory and motor planning in a natural task. , 2003, Journal of vision.

[20]  Fernanda Ferreira,et al.  Scene Perception for Psycholinguists. , 2004 .

[21]  Zenzi M. Griffin,et al.  Why Look? Reasons for Eye Movements Related to Language Production. , 2004 .

[22]  Naomi M. Kenner,et al.  How fast can you change your mind? The speed of top-down guidance in visual search , 2004, Vision Research.

[23]  J. Henderson,et al.  Prioritization of new objects in real-world scenes: evidence from eye movements. , 2005, Journal of experimental psychology. Human perception and performance.

[24]  L. Itti,et al.  Modeling the influence of task on attention , 2005, Vision Research.

[25]  J. Henderson,et al.  Object appearance, disappearance, and attention prioritization in real-world scenes , 2005, Psychonomic bulletin & review.

[26]  J. Henderson,et al.  Incidental visual memory for objects in scenes , 2005 .

[27]  Yuhong Jiang,et al.  Setting up the target template in visual search. , 2005, Journal of vision.

[28]  Miguel P Eckstein,et al.  Attentional Cues in Real Scenes, Saccadic Targeting, and Bayesian Priors , 2005, Psychological science.

[29]  P. Subramanian.  Active Vision: The Psychology of Looking and Seeing , 2006 .

[30]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[31]  T. Foulsham,et al.  How Does the Purpose of Inspection Influence the Potency of Visual Salience in Scene Perception? , 2007, Perception.

[32]  J. Henderson.  Regarding Scenes , 2007 .

[33]  Michael L. Mack,et al.  VISUAL SALIENCY DOES NOT ACCOUNT FOR EYE MOVEMENTS DURING VISUAL SEARCH IN REAL-WORLD SCENES , 2007 .

[34]  J. Henderson,et al.  Initial scene representations facilitate eye movement guidance in visual search. , 2007, Journal of experimental psychology. Human perception and performance.

[35]  John M. Henderson.  Eye Movements and Scene Memory , 2008 .

[36]  Jason M Haberman,et al.  Averaging facial expression over time. , 2009, Journal of vision.

[37]  Myriam Chanceaux,et al.  The influence of clutter on real-world scene search: evidence from search efficiency and eye movements. , 2009, Journal of vision.