Computational modelling of visual attention

Five important trends have emerged from recent work on computational models of focal visual attention that emphasize the bottom-up, image-based control of attentional deployment. First, the perceptual saliency of stimuli critically depends on the surrounding context. Second, a unique 'saliency map' that topographically encodes for stimulus conspicuity over the visual scene has proved to be an efficient and plausible bottom-up control strategy. Third, inhibition of return, the process by which the currently attended location is prevented from being attended again, is a crucial element of attentional deployment. Fourth, attention and eye movements tightly interplay, posing computational challenges with respect to the coordinate system used to control attention. And last, scene understanding and object recognition strongly constrain the selection of attended locations. Insights from these five key areas provide a framework for a computational and neurobiological understanding of visual attention.

[1]  W. James,et al.  The Principles of Psychology. , 1983 .

[2]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[3]  G. Schneider.  Two visual systems. , 1969, Science.

[4]  D. Noton,et al.  Eye movements and visual perception. , 1971, Scientific American.

[5]  RICHARD L. DIDDAY,et al.  Eye Movements and Visual Perception: A "Two Visual System" Model , 1975, Int. J. Man Mach. Stud..

[6]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[7]  P. Grobstein.  Analysis of Visual Behavior, David J. Ingle, Melvyn A. Goodale, Richard J.W. Mansfield (Eds.). MIT press, Cambridge, MA and London (1982), 834 , 1983 .

[8]  James R. Bergen,et al.  Parallel versus serial processing in rapid pattern discrimination , 1983, Nature.

[9]  J. Daugman.  Spatial visual channels in the fourier plane , 1984, Vision Research.

[10]  C. Mccoy.  The role of training. , 1984 .

[11]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[12]  T. Poggio,et al.  Spotlight on attention , 1985, Trends in Neurosciences.

[13]  J. Allman,et al.  Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local-global comparisons in visual neurons. , 1985, Annual review of neuroscience.

[14]  R. Desimone,et al.  Selective attention gates visual processing in the extrastriate cortex. , 1985, Science.

[15]  J. Findlay,et al.  The Relationship between Eye Movements and Spatial Attention , 1986, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[16]  G. Sperling,et al.  Dynamics of automatic and controlled visual attention. , 1987, Science.

[17]  A. Treisman.  Features and Objects: The Fourteenth Bartlett Memorial Lecture , 1988, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[18]  K. Nakayama,et al.  Sustained and transient components of focal visual attention , 1989, Vision Research.

[19]  T. Wiesel,et al.  Columnar specificity of intrinsic horizontal and corticocortical connections in cat visual cortex , 1989, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[20]  D. Sagi,et al.  Vision outside the focus of attention , 1990, Perception & psychophysics.

[21]  D. LaBerge,et al.  Positron emission tomographic measurements of pulvinar activity during an attention task , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[22]  L. Fogassi,et al.  Eye position effects on visual, memory, and saccade-related activity in areas LIP and 7a of macaque , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[23]  Michael S. Landy,et al.  Detection and Discrimination , 1991 .

[24]  S. Tipper,et al.  Short Report: Object-Centred Inhibition of Return of Visual Attention , 1991, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[25]  A. Leventhal.  The neural basis of visual function , 1991 .

[26]  Mark W. Cannon,et al.  Spatial interactions in apparent contrast: Inhibitory effects among grating patterns of different spatial frequencies, spatial positions and orientations , 1991, Vision Research.

[27]  I. Biederman,et al.  Dynamic binding in a neural network for shape recognition. , 1992, Psychological review.

[28]  I. Rock,et al.  Perceptual organization and attention , 1992, Cognitive Psychology.

[29]  S. Petersen,et al.  The pulvinar and visual salience , 1992, Trends in Neurosciences.

[30]  D. Sagi,et al.  Visual attention and perceptual grouping , 1992, Perception & psychophysics.

[31]  Peter Ford Dominey,et al.  A cortico-subcortical model for generation of spatially accurate sequential saccades. , 1992, Cerebral cortex.

[32]  H Egeth,et al.  Consequences of allocating attention to locations and to other attributes , 1992, Perception & psychophysics.

[33]  C. Koch,et al.  An oscillation-based model for the neuronal basis of attention , 1993, Vision Research.

[34]  J. Braun.  Shape-from-shading is independent of visual attention and may be a 'texton'. , 1993, Spatial vision.

[35]  Y. Burnod,et al.  Neural network models of cortical functions based on the computational properties of the cerebral cortex , 1994, Journal of Physiology - Paris.

[36]  Zijiang J. He,et al.  Perceiving textures: Beyond filtering , 1994, Vision Research.

[37]  J. Wolfe.  Visual search in continuous, naturalistic stimuli , 1994, Vision Research.

[38]  H. Egeth,et al.  Inhibition of return to object-based and environment-based locations , 1994, Perception & psychophysics.

[39]  M. Carandini,et al.  Summation and division by neurons in primate visual cortex. , 1994, Science.

[40]  B. Motter.  Neural correlates of attentive selection for color or luminance in extrastriate area V4 , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[41]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[42]  H. Jones,et al.  Visual cortical mechanisms detecting focal orientation discontinuities , 1995, Nature.

[43]  N. Logothetis,et al.  Shape representation in the inferior temporal cortex of monkeys , 1995, Current Biology.

[44]  John K. Tsotsos,et al.  Modeling Visual Attention via Selective Tuning , 1995, Artif. Intell..

[45]  Thierry Pun,et al.  Attentive mechanisms for dynamic and static scene analysis , 1995 .

[46]  J. Hoffman,et al.  The role of visual attention in saccadic eye movements , 1995, Perception & psychophysics.

[47]  B. Dosher,et al.  The role of attention in the programming of saccades , 1995, Vision Research.

[48]  D. Robinson,et al.  Shared neural control of attentional shifts and eye movements , 1996, Nature.

[49]  John H. R. Maunsell,et al.  Attentional modulation of visual motion processing in cortical areas MT and MST , 1996, Nature.

[50]  Pietro Perona,et al.  Early computation of shape and reflectance in the visual system , 1996, Nature.

[51]  C. Freksa,et al.  Visual Attention and Cognition , 1996 .

[52]  S Shimojo,et al.  Orienting a spatial attention--its reflexive, compensatory, and voluntary mechanisms. , 1996, Brain research. Cognitive brain research.

[53]  S. Shimojo,et al.  Stimulus-driven facilitation and inhibition of visual information processing in environmental and retinotopic representations of space. , 1996, Brain research. Cognitive brain research.

[54]  A. Treisman,et al.  Visual memory for novel shapes: implicit coding without attention. , 1996, Journal of experimental psychology. Learning, memory, and cognition.

[55]  L. Stark,et al.  Experimental metaphysics: The scanpath as an epistemological mechanism , 1996 .

[56]  H. Egeth,et al.  Perception without attention: evidence of grouping under conditions of inattention. , 1997, Journal of experimental psychology. Human perception and performance.

[57]  V. Lakshminarayanan,et al.  Basic and Clinical Applications of Vision Science , 1997, Documenta Ophthalmologica Proceedings Series.

[58]  H. Egeth,et al.  Perception without attention: evidence of grouping under conditions of inattention. , 1997, Journal of Experimental Psychology: Human Perception and Performance.

[59]  J. B. Levitt,et al.  Contrast dependence of contextual effects in primate visual cortex , 1997, nature.

[60]  T. Sejnowski,et al.  Spatial Transformations in the Parietal Cortex Using Basis Functions , 1997, Journal of Cognitive Neuroscience.

[61]  Gerhard Krieger,et al.  Investigation of a sensorimotor system for saccadic scene analysis: an integrated approach , 1998 .

[62]  C. Koch,et al.  Constraints on cortical and thalamic projections: the no-strong-loops hypothesis , 1998, Nature.

[63]  Pieter R. Roelfsema,et al.  Object-based attention in the primary visual cortex of the macaque monkey , 1998, Nature.

[64]  I. Rentschler,et al.  Intrinsic two-dimensional features as textons. , 1998, Journal of the Optical Society of America. A, Optics, image science, and vision.

[65]  Todd S. Horowitz,et al.  Visual search has no memory , 1998, Nature.

[66]  M Corbetta,et al.  Frontoparietal cortical networks for directing attention and the eye to visual locations: identical, independent, or overlapping neural systems? , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[67]  Marisa Carrasco,et al.  Attention improves or impairs visual performance by enhancing spatial resolution , 1998, Nature.

[68]  B. C. Motter,et al.  The guidance of eye movements during active visual search , 1998, Vision Research.

[69]  M. Goldberg,et al.  The representation of visual salience in monkey parietal cortex , 1998, Nature.

[70]  R. Parasuraman.  The attentive brain , 1998 .

[71]  J. Braun.  Vision and attention: the role of training , 1998, Nature.

[72]  B. Julesz,et al.  Withdrawing attention at little or no cost: Detection and discrimination tasks , 1998, Perception & psychophysics.

[73]  I. Rybak,et al.  A model of attention-guided visual perception and recognition , 1998, Vision Research.

[74]  E. DeYoe,et al.  A physiological correlate of the 'spotlight' of visual attention , 1999, Nature Neuroscience.

[75]  Stefan Treue,et al.  Feature-based attention influences motion processing gain in macaque visual cortex , 1999, Nature.

[76]  G Sperling,et al.  Measuring the amplification of attention. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[77]  I. Biederman,et al.  Localizing the cortical region mediating visual awareness of object identity. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[78]  P Reinagel,et al.  Natural scene statistics at the centre of gaze. , 1999, Network.

[79]  R. Rafal,et al.  Components of reflexive visual orienting to moving objects , 1999, Perception & psychophysics.

[80]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[81]  R. Desimone,et al.  The Role of Neural Mechanisms of Attention in Solving the Binding Problem , 1999, Neuron.

[82]  J. Pratt,et al.  Inhibition of return is composed of attentional and oculomotor processes , 1999, Perception & psychophysics.

[83]  C. Connor,et al.  Responses to contour features in macaque area V4. , 1999, Journal of neurophysiology.

[84]  Nancy Kanwisher,et al.  fMRI evidence for objects as the units of attentional selection , 1999, Nature.

[85]  C. Koch,et al.  Attention activates winner-take-all competition among visual filters , 1999, Nature Neuroscience.

[86]  Alexander Toet,et al.  A high-resolution image data set for testing search and detection models , 1999 .

[87]  Iain D Gilchrist,et al.  Saccade selection in visual search: evidence for spatial frequency specific between-item interactions , 1999, Vision Research.

[88]  Karl J. Friston,et al.  The physiological basis of attentional modulation in extrastriate visual areas , 1999, Nature Neuroscience.

[89]  M. Goldberg,et al.  Space and attention in parietal cortex. , 1999, Annual review of neuroscience.

[90]  R A Abrams,et al.  Object-based visual attention with endogenous orienting , 2000, Perception & psychophysics.

[91]  D. Gitelman,et al.  Covert Visual Spatial Orienting and Saccades: Overlapping Neural Systems , 2000, NeuroImage.

[92]  C. Gilbert,et al.  Interactions between attention, context and learning in primary visual cortex , 2000, Vision Research.

[93]  R. Klein,et al.  Visual and motor effects in inhibition of return. , 2000, Journal of experimental psychology. Human perception and performance.

[94]  G. Deco,et al.  A hierarchical neural system with attentional top–down enhancement of the spatial resolution for object recognition , 2000, Vision Research.

[95]  J L Gallant,et al.  Sparse coding and decorrelation in primary visual cortex during natural vision. , 2000, Science.

[96]  M. Corbetta,et al.  Voluntary orienting is dissociated from target detection in human posterior parietal cortex , 2000, Nature Neuroscience.

[97]  H. Nothdurft.  Salience from feature contrast: additivity across dimensions , 2000, Vision Research.

[98]  R. Knight,et al.  Prefrontal modulation of visual processing in humans , 2000, Nature Neuroscience.

[99]  M. Carrasco,et al.  Spatial covert attention increases contrast sensitivity across the CSF: support for signal enhancement , 2000, Vision Research.

[100]  C. Koch,et al.  Category-specific visual responses of single neurons in the human medial temporal lobe , 2000, Nature Neuroscience.

[101]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[102]  C Koch,et al.  Revisiting spatial vision: toward a unifying model. , 2000, Journal of the Optical Society of America. A, Optics, image science, and vision.

[103]  H Egeth,et al.  Mixed reference frames for dynamic inhibition of return. , 2000, Journal of experimental psychology. Human perception and performance.

[104]  E. Miller,et al.  THE PREFRONTAL CORTEX AND COGNITIVE CONTROL , 2000 .

[105]  B. Dosher,et al.  Mechanisms of perceptual attention in precuing of location , 2000, Vision Research.

[106]  J. Schall,et al.  Antecedents and correlates of visual detection and awareness in macaque prefrontal cortex , 2000, Vision Research.

[107]  Tomaso Poggio,et al.  Models of object recognition , 2000, Nature Neuroscience.

[108]  S. Grossberg,et al.  Contrast-sensitive perceptual grouping and object-based attention in the laminar circuits of primary visual cortex , 2000, Vision Research.

[109]  E. Miller,et al.  The prefontral cortex and cognitive control , 2000, Nature Reviews Neuroscience.

[110]  Dave Mercer,et al.  The role of training , 2000 .

[111]  G. Mangun,et al.  The neural mechanisms of top-down attentional control , 2000, Nature Neuroscience.

[112]  J. Schall,et al.  Neural control of behavior: countermanding eye movements , 2000, Psychological research.

[113]  R. Desimone,et al.  Attention Increases Sensitivity of V4 Neurons , 2000, Neuron.

[114]  Shaul Hochstein,et al.  The spread of attention and learning in feature search: effects of target distribution and task difficulty , 2000, Vision Research.

[115]  C. Gilbert,et al.  Learning to find a shape , 2000, Nature Neuroscience.

[116]  K Suder,et al.  The Control of Low-Level Information Flow in the Visual System , 2000, Reviews in the neurosciences.

[117]  R. Desimone,et al.  Attention Increases Sensitivity of V4 Neurons , 2000, Neuron.

[118]  Claudio M. Privitera,et al.  Representation of human vision in the brain: How does human perception recognize images? , 2001, J. Electronic Imaging.

[119]  Christof Koch,et al.  Feature combination strategies for saliency-based visual attention systems , 2001, J. Electronic Imaging.

[120]  J. C. Johnston,et al.  Attention and performance. , 2001, Annual review of psychology.

[121]  Author Update , 2001, Neurology.

[122]  I. Biederman,et al.  Inferior Temporal Neurons Show Greater Sensitivity to Nonaccidental than to Metric Shape Differences , 2001, Journal of Cognitive Neuroscience.

[123]  Gerhard Krieger,et al.  Scene analysis with saccadic eye movements: Top-down and bottom-up modeling , 2001, J. Electronic Imaging.