A model of top-down attentional control during visual search in complex scenes.

Recently, there has been great interest among vision researchers in developing computational models that predict the distribution of saccadic endpoints in naturalistic scenes. In many of these studies, subjects are instructed to view scenes without any particular task in mind so that stimulus-driven (bottom-up) processes guide visual attention. However, whenever there is a search task, goal-driven (top-down) processes tend to dominate guidance, as indicated by attention being systematically biased toward image features that resemble those of the search target. In the present study, we devise a top-down model of visual attention during search in complex scenes based on similarity between the target and regions of the search scene. Similarity is defined for several feature dimensions such as orientation or spatial frequency using a histogram-matching technique. The amount of attentional guidance across visual feature dimensions is predicted by a previously introduced informativeness measure. We use eye-movement data gathered from participants' search of a set of naturalistic scenes to evaluate the model. The model is found to predict the distribution of saccadic endpoints in search displays nearly as accurately as do other observers' eye-movement data in the same displays.

[1]  Amelia Johannes,et al.  PERIPHERAL VISION. , 1929, British medical journal.

[2]  H.M. Wechsler,et al.  Digital image processing, 2nd ed. , 1981, Proceedings of the IEEE.

[3]  P. Lennie,et al.  Chromatic mechanisms in lateral geniculate nucleus of macaque. , 1984, The Journal of physiology.

[4]  Paul Wintz,et al.  Digital image processing (2nd ed.) , 1987 .

[5]  J. Duncan,et al.  Visual search and stimulus similarity. , 1989, Psychological review.

[6]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[7]  P. Lennie,et al.  Chromatic mechanisms in striate cortex of macaque , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[8]  J. Wolfe,et al.  Guided Search 2.0 A revised model of visual search , 1994, Psychonomic bulletin & review.

[9]  M. H. Brill,et al.  How the CIE 1931 color-matching functions were derived from Wright-Guild data , 1997 .

[10]  Michael H. Brill,et al.  Color appearance models , 1998 .

[11]  C. Erkelens,et al.  Peripheral vision and oculomotor control during visual search , 1999, Vision Research.

[12]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[13]  M. Pomplun,et al.  Distractor Ratio Influences Patterns of Eye Movements during Visual Search , 2000, Perception.

[14]  Bevil R. Conway,et al.  Spatial Structure of Cone Inputs to Color Cells in Alert Macaque Primary Visual Cortex (V-1) , 2001, The Journal of Neuroscience.

[15]  D E Williams,et al.  Preattentive guidance of eye movements during triple conjunction search tasks: The effects of feature discriminability and saccadic amplitude , 2001, Psychonomic bulletin & review.

[16]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[17]  H J Müller,et al.  Top-down controlled visual dimension weighting: an event-related fMRI study. , 2002, Cerebral cortex.

[18]  Derrick J. Parkhurst,et al.  Modeling the role of salience in the allocation of overt visual attention , 2002, Vision Research.

[19]  J. Henderson.  Human gaze control during real-world scene perception , 2003, Trends in Cognitive Sciences.

[20]  Eyal M. Reingold,et al.  Area activation: a computational model of saccadic selectivity in visual search , 2003, Cogn. Sci..

[21]  P. Quinlan.  Visual feature integration theory: past, present, and future. , 2003, Psychological bulletin.

[22]  Helga C. Arsenio,et al.  Panoramic search: the interaction of memory and vision in search through a familiar scene. , 2004, Journal of experimental psychology. Human perception and performance.

[23]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[24]  David J. Hand,et al.  A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems , 2001, Machine Learning.

[25]  Zenzi M. Griffin,et al.  Why Look? Reasons for Eye Movements Related to Language Production. , 2004 .

[26]  Wilson S. Geisler,et al.  Optimal eye movement strategies in visual search , 2005, Nature.

[27]  Wei Zhang,et al.  The Role of Top-down and Bottom-up Processes in Guiding Eye Movements during Visual Search , 2005, NIPS.

[28]  John K. Tsotsos,et al.  Saliency Based on Information Maximization , 2005, NIPS.

[29]  Simone Frintrop,et al.  Goal-Directed Search with a Top-Down Modulated Computational Attention System , 2005, DAGM-Symposium.

[30]  Iain D. Gilchrist,et al.  Visual correlates of fixation selection: effects of scale and time , 2005, Vision Research.

[31]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[32]  Laurent Itti,et al.  An Integrated Model of Top-Down and Bottom-Up Attention for Optimizing Detection Speed , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[33]  M. Pomplun.  Saccadic selectivity in complex visual search displays , 2006, Vision Research.

[34]  Gregory J. Zelinsky,et al.  Scene context guides eye movements during visual search , 2006, Vision Research.

[35]  Alex D. Hwang,et al.  How Chromaticity Guides Visual Search in Real-World Scenes , 2007 .

[36]  C. Koch,et al.  Probabilistic modeling of eye movement data during conjunction search via feature-based attention. , 2007, Journal of vision.

[37]  Robin L. Hill,et al.  Eye movements : a window on mind and brain , 2007 .

[38]  B. C. Motter,et al.  Saccades and covert shifts of attention during active visual search: Spatial distributions, memory, and items per fixation , 2007, Vision Research.

[39]  L. Itti,et al.  Search Goal Tunes Visual Features Optimally , 2007, Neuron.

[40]  Laurent Itti,et al.  Beyond bottom-up: Incorporating task-dependent influences into a computational model of spatial attention , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  D. McNamara,et al.  Proceedings of the 29th Annual Cognitive Science Society , 2007 .

[42]  Benjamin W Tatler,et al.  The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. , 2007, Journal of vision.

[43]  Michael L. Mack,et al.  VISUAL SALIENCY DOES NOT ACCOUNT FOR EYE MOVEMENTS DURING VISUAL SEARCH IN REAL-WORLD SCENES , 2007 .

[44]  Jim M. Monti,et al.  Neural Integration of Top-Down Spatial and Feature-Based Information in Visual Search , 2008, The Journal of Neuroscience.

[45]  Alex D. Hwang,et al.  Informativeness of visual features guides search , 2008 .

[46]  G. Zelinsky.  A theory of eye movements during target acquisition. , 2008, Psychological review.