Salient object changes influence overt attentional prioritization and object-based targeting in natural scenes

A change to an object in natural scenes attracts attention when it occurs during a fixation. However, when a change occurs during a saccade, and is masked by saccadic suppression, it typically does not capture the gaze in a bottom-up manner. In the present work, we investigated how the type and direction of salient changes to objects affect the prioritization and targeting of objects in natural scenes. We asked observers to look around a scene in preparation for a later memory test. After a period of time, an object in the scene was increased or decreased in salience either during a fixation (with a transient signal) or during a saccade (without transient signal), or it was not changed at all. Changes that were made during a fixation attracted the eyes both when the change involved an increase and a decrease in salience. However, changes that were made during a saccade only captured the eyes when the change was an increase in salience, relative to the baseline no-change condition. These results suggest that the prioritization of object changes can be influenced by the underlying salience of the changed object. In addition, object changes that occurred with a transient signal (which is itself a salient signal) resulted in more central object targeting. Taken together, our results suggest that salient signals in a natural scene are an important component in both object prioritization and targeting in natural scene viewing, insofar as they align with object locations.

[1]  C. Koch,et al.  Task-demands can immediately reverse the effects of sensory-driven saliency in complex visual stimuli. , 2008, Journal of vision.

[2]  J. Theeuwes,et al.  Programming of endogenous and exogenous saccades: evidence for a competitive integration model. , 2002, Journal of experimental psychology. Human perception and performance.

[3]  Michael L. Mack,et al.  VISUAL SALIENCY DOES NOT ACCOUNT FOR EYE MOVEMENTS DURING VISUAL SEARCH IN REAL-WORLD SCENES , 2007 .

[4]  T. Foulsham,et al.  Eye movements during scene inspection: A test of the saliency map hypothesis , 2006 .

[5]  Robert J. Moorhead,et al.  Visualization of fluid flows in virtual environments , 2004, J. Vis..

[6]  Debasish Biswas,et al.  Visualization of unsteady viscous flow around turbine blade , 2008, J. Vis..

[7]  D. S. Wooding,et al.  The relationship between the locations of spatial features and those of fixations made during visual examination of briefly presented images. , 1996, Spatial vision.

[8]  P. Perona,et al.  Rapid natural scene categorization in the near absence of attention , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Patrick Cavanagh,et al.  Allocation of attention across saccades. , 2013, Journal of neurophysiology.

[10]  J. Henderson,et al.  Object appearance, disappearance, and attention prioritization in real-world scenes , 2005, Psychonomic bulletin & review.

[11]  R. Klein,et al.  A Model of Saccade Initiation Based on the Competitive Integration of Exogenous and Endogenous Signals in the Superior Colliculus , 2001, Journal of Cognitive Neuroscience.

[12]  K. Fujii,et al.  Visualization for the analysis of fluid motion , 2005, J. Vis..

[13]  D. Ballard,et al.  Eye movements in natural behavior , 2005, Trends in Cognitive Sciences.

[14]  A. Jacobs,et al.  Optimal viewing position effect in word recognition: A challenge to current theory. , 1992 .

[15]  D. E. Irwin,et al.  Our Eyes do Not Always Go Where we Want Them to Go: Capture of the Eyes by New Objects , 1998 .

[16]  Denis Cousineau,et al.  Confidence intervals in within-subject designs: A simpler solution to Loftus and Masson's method , 2005 .

[17]  Nicola C. Anderson,et al.  The influence of a scene preview on eye movement behavior in natural scenes , 2016, Psychonomic bulletin & review.

[18]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[19]  W. Einhäuser,et al.  Fixations on objects in natural scenes: dissociating importance from salience , 2013, Front. Psychol..

[20]  Jan Theeuwes,et al.  OpenSesame: An open-source, graphical experiment builder for the social sciences , 2011, Behavior Research Methods.

[21]  R. Rafal,et al.  Shifting visual attention between objects and locations: evidence from normal and parietal lesion subjects. , 1994, Journal of experimental psychology. General.

[22]  B. Ripley,et al.  Random and Mixed Effects , 2002 .

[23]  H. Müller,et al.  Salience-Based Selection: Attentional Capture by Distractors Less Salient Than the Target , 2013, PloS one.

[24]  Tom Foulsham,et al.  Optimal and Preferred Eye Landing Positions in Objects and Scenes , 2013, Quarterly journal of experimental psychology.

[25]  Derrick J. Parkhurst,et al.  Modeling the role of salience in the allocation of overt visual attention , 2002, Vision Research.

[26]  A. Treisman,et al.  Perception of objects in natural scenes: is it really attention free? , 2005, Journal of experimental psychology. Human perception and performance.

[27]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[28]  T. Foulsham,et al.  Saliency and scan patterns in the inspection of real-world scenes: Eye movements during encoding and recognition , 2009 .

[29]  J. Theeuwes,et al.  The role of stimulus-driven and goal-driven control in saccadic visual selection. , 2004, Journal of experimental psychology. Human perception and performance.

[30]  J. Henderson,et al.  Prioritizing new objects for eye fixation in real-world scenes: Effects of object–scene consistency , 2008 .

[31]  S. Yantis,et al.  Uniqueness of abrupt visual onset in capturing attention , 1988, Perception & psychophysics.

[32]  T. Foulsham,et al.  What can saliency models predict about eye movements? Spatial and sequential aspects of fixations during encoding and recognition. , 2008, Journal of vision.

[33]  P. Perona,et al.  Objects predict fixations better than early saliency. , 2008, Journal of vision.

[34]  D. Ballard,et al.  Eye guidance in natural vision: reinterpreting salience. , 2011, Journal of vision.

[35]  N. Charness,et al.  Visual Span in Expert Chess Players: Evidence From Eye Movements , 2001, Psychological science.

[36]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[37]  Iain D. Gilchrist,et al.  Visual correlates of fixation selection: effects of scale and time , 2005, Vision Research.

[38]  R. Baddeley,et al.  The long and the short of it: Spatial statistics at fixation vary with saccade amplitude and task , 2006, Vision Research.

[39]  Richard D. Morey,et al.  Confidence Intervals from Normalized Data: A correction to Cousineau (2005) , 2008 .

[40]  M. Donk,et al.  Object Salience is Transiently Represented Whereas Object Presence is Not: Evidence from Temporal Order Judgment , 2011, Perception.

[41]  K. Rayner.  Eye Guidance in Reading: Fixation Locations within Words , 1979, Perception.

[42]  Nicola C. Anderson,et al.  It depends on when you look at it: Salience influences eye movements in natural scene viewing and search early in time. , 2015, Journal of vision.

[43]  D. Simons,et al.  Do New Objects Capture Attention? , 2005, Psychological science.

[44]  A. Belopolsky,et al.  Target–Distractor Competition in the Oculomotor System Is Spatiotopic , 2014, The Journal of Neuroscience.

[45]  Gary S. Settles,et al.  Full-scale schlieren visualization of supersonic bullet and muzzle blast from firing a .30-06 rifle , 2005, J. Vis..

[46]  Françoise Vitu,et al.  Dynamic coding of saccade length in reading , 2007 .

[47]  P. König,et al.  Does luminance‐contrast contribute to a saliency map for overt visual attention? , 2003, The European journal of neuroscience.

[48]  L. Zhaoping.  Attention capture by eye of origin singletons even without awareness--a hallmark of a bottom-up saliency map in the primary visual cortex. , 2008, Journal of vision.

[49]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[50]  J. Henderson,et al.  Oculomotor capture during real-world scene viewing depends on cognitive load , 2011, Vision Research.

[51]  J. Henderson,et al.  Object-based attentional selection in scene viewing. , 2010, Journal of vision.

[52]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[53]  M. Tinker.  How People Look at Pictures. , 1936 .

[54]  M. Donk,et al.  The effects of saccade-contingent changes on oculomotor capture: salience is important even beyond the first oculomotor response , 2014, Attention, perception & psychophysics.

[55]  Mary M Hayhoe,et al.  Task and context determine where you look. , 2016, Journal of vision.

[56]  J. Henderson,et al.  Overt attentional prioritization of new objects and feature changes during real-world scene viewing , 2009 .

[57]  G. Altmann.  Language-mediated eye movements in the absence of a visual world: the ‘blank screen paradigm’ , 2004, Cognition.

[58]  George L. Malcolm,et al.  Searching in the dark: Cognitive relevance drives attention in real-world scenes , 2009, Psychonomic bulletin & review.

[59]  Barry D. Vaughan,et al.  Object-Based Visual Selection: Evidence From Perceptual Completion , 1998 .

[60]  Pedro M. Valero-Mora,et al.  ggplot2: Elegant Graphics for Data Analysis , 2010 .

[61]  Christof Koch,et al.  Modeling attention to salient proto-objects , 2006, Neural Networks.

[62]  A. L. Yarbus.  Eye Movements During Perception of Complex Objects , 1967 .

[63]  Maciej Pajak,et al.  Object-based saccadic selection during scene perception: evidence from viewing position effects. , 2013, Journal of vision.

[64]  G. Underwood,et al.  Low-level visual saliency does not predict change detection in natural scenes. , 2007, Journal of vision.

[65]  Wieske van Zoest,et al.  Oculomotor Evidence for Top-Down Control following the Initial Saccade , 2011, PloS one.

[66]  S. Yantis,et al.  New objects dominate luminance transients in setting attentional priority. , 2001, Journal of experimental psychology. Human perception and performance.

[67]  Hadley Wickham,et al.  ggplot2 - Elegant Graphics for Data Analysis (2nd Edition) , 2017 .

[68]  M. Pomplun,et al.  The roles of scene gist and spatial dependency among objects in the semantic guidance of attention in real-world scenes , 2014, Vision Research.