Attention in natural scenes: Neurophysiological and computational bases

How does attention operate in natural scenes? We show that the receptive fields of inferior temporal cortex neurons that implement object representations become small and located at the fovea in complex natural scenes. This facilitates the readout of information about an object that may be reward or punishment associated, and may be the target for action. Top-down biased competition to implement attention has a much weaker effect in complex natural scenes than in otherwise blank scenes with two objects. Part of the solution to the binding problem is thus that competition and the foveal cortical magnification factor emphasize what is present at the fovea, and limit the binding problem. Part of the solution to the binding problem is that neurons respond to combinations of features present in the correct relative spatial positions. Stimulus-dependent neuronal synchrony does not appear to be quantitatively important in feature binding, and in attention, in natural visual scenes, at least in the inferior temporal visual cortex, as shown by information theoretic analyses. The perception of multiple objects in a scene is facilitated by the fact that inferior temporal visual cortex neurons have asymmetrical receptive fields with respect to the fovea in complex scenes. Computational models of this processing are described.

[1]  E. Rolls,et al.  Object‐based visual neglect: a computational hypothesis , 2002, The European journal of neuroscience.

[2]  E T Rolls,et al.  Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex. , 1995, Journal of neurophysiology.

[3]  E. Miller,et al.  Suppression of visual responses of neurons in inferior temporal cortex of the awake macaque by addition of a second stimulus , 1993, Brain Research.

[4]  G. Deco,et al.  Top-down selective visual attention: A neurodynamical approach , 2001 .

[5]  M. Tovée,et al.  Information encoding in short firing rate epochs by single neurons in the primate temporal visual cortex , 1995 .

[6]  E. Rolls,et al.  Attention and working memory: a dynamical model of neuronal activity in the prefrontal cortex , 2003, The European journal of neuroscience.

[7]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[8]  G. Deco,et al.  Cooperation and biased competition model can explain attentional filtering in the prefrontal cortex , 2004, The European journal of neuroscience.

[9]  Leonardo Franco,et al.  The use of decoding to analyze the contribution to the information of the correlations between the firing of simultaneously recorded neurons , 2004, Experimental Brain Research.

[10]  E. Rolls,et al.  Synaptic and spiking dynamics underlying reward reversal in the orbitofrontal cortex. , 2004, Cerebral cortex.

[11]  M. Gazzaniga,et al.  The new cognitive neurosciences , 2000 .

[12]  A. Treves,et al.  The representational capacity of the distributed encoding of information provided by populations of neurons in primate temporal visual cortex , 1997, Experimental Brain Research.

[13]  Edmund T. Rolls,et al.  Information encoding in the inferior temporal visual cortex: contributions of the firing rates and the correlations between the firing of neurons , 2010, Biological Cybernetics.

[14]  M. Gazzaniga.  The new cognitive neurosciences, 2nd ed. , 2000 .

[15]  M. Tovée,et al.  Translation invariance in the responses to faces of single neurons in the temporal visual cortical areas of the alert macaque. , 1994, Journal of neurophysiology.

[16]  E. Rolls,et al.  The Neurophysiology of Backward Visual Masking: Information Analysis , 1999, Journal of Cognitive Neuroscience.

[17]  H. Spitzer,et al.  Increased attention enhances both behavioral and neuronal performance. , 1988, Science.

[18]  R. Desimone,et al.  The Role of Neural Mechanisms of Attention in Solving the Binding Problem , 1999, Neuron.

[19]  A. Cowey,et al.  Topography of the retina and striate cortex and its relationship to visual acuity in rhesus monkeys and squirrel monkeys , 2004, Experimental Brain Research.

[20]  Edmund T Rolls,et al.  Consciousness absent and present: a neurophysiological exploration. , 2004, Progress in brain research.

[21]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[22]  E T Rolls,et al.  Invariant object recognition in the visual system with error correction and temporal difference learning , 2001, Network.

[23]  E. Rolls,et al.  INVARIANT FACE AND OBJECT RECOGNITION IN THE VISUAL SYSTEM , 1997, Progress in Neurobiology.

[24]  Edmund T. Rolls,et al.  Invariant recognition of feature combinations in the visual system , 2002, Biological Cybernetics.

[25]  Edmund T. Rolls,et al.  Invariant Global Motion Recognition in the Dorsal Visual System: A Unifying Theory , 2007, Neural Computation.

[26]  S. Treue,et al.  Attentional Modulation Strength in Cortical Area MT Depends on Stimulus Contrast , 2002, Neuron.

[27]  Steven Reece,et al.  An information theoretic approach to the contributions of the firing rates and the correlations between the firing of neurons. , 2003, Journal of neurophysiology.

[28]  G. Deco,et al.  The time course of selective visual attention: theory and experiments , 2002, Vision Research.

[29]  M. Tovée,et al.  Information encoding and the responses of single neurons in the primate temporal visual cortex. , 1993, Journal of neurophysiology.

[30]  E T Rolls,et al.  Neurophysiological mechanisms underlying face processing within and beyond the temporal cortical visual areas. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[31]  E. Rolls,et al.  What and Where in Visual Working Memory: A Computational Neurodynamical Perspective for Integrating fMRI and Single-Neuron Data , 2004, Journal of Cognitive Neuroscience.

[32]  Edmund T. Rolls,et al.  Position invariant recognition in the visual system with cluttered environments , 2000, Neural Networks.

[33]  James L. McGaugh,et al.  Brain Organization and Memory: Cells, Systems, and Circuits , 1992 .

[34]  M. Tovée,et al.  The responses of neurons in the temporal cortex of primates, and face identification and detection , 1994, Experimental Brain Research.

[35]  Edmund T Rolls,et al.  The Receptive Fields of Inferior Temporal Cortex Neurons in Natural Scenes , 2003, The Journal of Neuroscience.

[36]  Thomas P. Trappenberg,et al.  Effective Size of Receptive Fields of Inferior Temporal Visual Cortex Neurons in Natural Scenes , 2001, NIPS.

[37]  Tai Sing Lee,et al.  A unified model of spatial and object attention based on inter-cortical biased competition , 2002, Neurocomputing.

[38]  P König,et al.  Formation of cortical cell assemblies. , 1990, Cold Spring Harbor symposia on quantitative biology.

[39]  Ch. von der Malsburg,et al.  A neural cocktail-party processor , 1986, Biological Cybernetics.

[40]  E. Rolls,et al.  Decision‐making and Weber's law: a neurophysiological model , 2006, The European journal of neuroscience.

[41]  M. Tovée,et al.  Processing speed in the cerebral cortex and the neurophysiology of visual masking , 1994, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[42]  I. Biederman.  Perceiving Real-World Scenes , 1972, Science.

[43]  J. Duncan.  Cooperating brain systems in selective perception and action. , 1996 .

[44]  Wolf Singer,et al.  Neuronal Synchrony: A Versatile Code for the Definition of Relations? , 1999, Neuron.

[45]  Edmund T. Rolls,et al.  Invariant visual object recognition: A model, with lighting invariance , 2006, Journal of Physiology-Paris.

[46]  E. Rolls,et al.  Scene perception: inferior temporal cortex neurons encode the positions of different objects in the scene , 2005, The European journal of neuroscience.

[47]  E. Rolls,et al.  Neurodynamics of biased competition and cooperation for attention: a model with spiking neurons. , 2005, Journal of neurophysiology.

[48]  R. Desimone,et al.  Competitive Mechanisms Subserve Attention in Macaque Areas V2 and V4 , 1999, The Journal of Neuroscience.

[49]  Gustavo Deco,et al.  Large-scale neural model for visual attention: integration of experimental single-cell and fMRI data. , 2002, Cerebral cortex.

[50]  Edmund T. Rolls,et al.  Invariant Object Recognition in the Visual System with Novel Views of 3D Objects , 2002, Neural Computation.

[51]  A. Cowey,et al.  Human cortical magnification factor and its relation to visual acuity , 2004, Experimental Brain Research.

[52]  Michael C. Mozer,et al.  Perception of multiple objects - a connectionist approach , 1991, Neural network modeling and connectionism.

[53]  R. Desimone,et al.  Selective attention gates visual processing in the extrastriate cortex. , 1985, Science.

[54]  Gustavo Deco,et al.  Computational neuroscience of vision , 2002 .

[55]  John Duncan,et al.  A neural basis for visual search in inferior temporal cortex , 1993, Nature.

[56]  Stefano Panzeri,et al.  Information in the Neuronal Representation of Individual Stimuli in the Primate Temporal Visual Cortex , 1997, Journal of Computational Neuroscience.

[57]  E. Rolls,et al.  Size and contrast have only small effects on the responses to faces of neurons in the cortex of the superior temporal sulcus of the monkey , 2004, Experimental Brain Research.

[58]  I. Biederman,et al.  Dynamic binding in a neural network for shape recognition. , 1992, Psychological review.

[59]  B. C. Motter.  Focal attention produces spatially selective processing in visual cortical areas V1, V2, and V4 in the presence of competing stimuli. , 1993, Journal of neurophysiology.

[60]  E. Niebur,et al.  Modeling the Temporal Dynamics of IT Neurons in Visual Search: A Mechanism for Top-Down Selective Attention , 1996, Journal of Cognitive Neuroscience.

[61]  Edmund T. Rolls,et al.  Learning invariant object recognition in the visual system with continuous transformations , 2006, Biological Cybernetics.

[62]  M. Hasselmo,et al.  The role of expression and identity in the face-selective responses of neurons in the temporal visual cortex of the monkey , 1989, Behavioural Brain Research.

[63]  E. Rolls,et al.  Attention, short-term memory, and action selection: A unifying theory , 2005, Progress in Neurobiology.

[64]  E. Rolls,et al.  Information in the first spike, the order of spikes, and the number of spikes provided by neurons in the inferior temporal visual cortex , 2006, Vision Research.

[65]  E. Rolls,et al.  View-invariant representations of familiar objects by neurons in the inferior temporal visual cortex. , 1998, Cerebral cortex.

[66]  E. Rolls,et al.  A Neurodynamical cortical model of visual attention and invariant object recognition , 2004, Vision Research.

[67]  Edmund T. Rolls,et al.  Functions of the Primate Temporal Lobe Cortical Visual Areas in Invariant Visual Object and Face Recognition , 2000, Neuron.

[68]  E. Rolls,et al.  Object perception in natural scenes: encoding by inferior temporal cortex simultaneously recorded neurons. , 2005, Journal of neurophysiology.

[69]  E. Rolls.  The representation of information about faces in the temporal and frontal lobes , 2007, Neuropsychologia.

[70]  L. Chelazzi.  Serial attention mechanisms in visual search: A critical look at the evidence , 1999, Psychological research.

[71]  Edmund T. Rolls,et al.  A Model of Invariant Object Recognition in the Visual System: Learning Rules, Activation Functions, Lateral Inhibition, and Information-Based Performance Measures , 2000, Neural Computation.

[72]  W Singer,et al.  Visual feature integration and the temporal correlation hypothesis. , 1995, Annual review of neuroscience.

[73]  Gustavo Deco,et al.  Feature-based Attention in Human Visual Cortex: Simulation of Fmri Data , 2003 .