Invariant visual object recognition: A model, with lighting invariance

How are invariant representations of objects formed in the visual cortex? We describe a neurophysiological and computational approach which focusses on a feature hierarchy model in which invariant representations can be built by self-organizing learning based on the statistics of the visual input. The model can use temporal continuity in an associative synaptic learning rule with a short term memory trace, and/or it can use spatial continuity in Continuous Transformation learning. The model of visual processing in the ventral cortical stream can build representations of objects that are invariant with respect to translation, view, size, and in this paper we show also lighting. The model has been extended to provide an account of invariant representations in the dorsal visual system of the global motion produced by objects such as looming, rotation, and object-based movement. The model has been extended to incorporate top-down feedback connections to model the control of attention by biased competition in for example spatial and object search tasks. The model has also been extended to account for how the visual system can select single objects in complex visual scenes, and how multiple objects can be represented in a scene.

[1]  M. Graziano,et al.  Tuning of MST neurons to spiral motions , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[2]  Moshe Abeles,et al.  Corticonics: Neural Circuits of Cerebral Cortex , 1991 .

[3]  M. Tovée,et al.  Processing speed in the cerebral cortex and the neurophysiology of visual masking , 1994, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[4]  I. Biederman Perceiving Real-World Scenes , 1972, Science.

[5]  D C Van Essen,et al.  Information processing in the primate visual system: an integrated systems perspective. , 1992, Science.

[6]  J. Duncan Cooperating brain systems in selective perception and action. , 1996 .

[7]  Wolf Singer,et al.  Neuronal Synchrony: A Versatile Code for the Definition of Relations? , 1999, Neuron.

[8]  I. Biederman,et al.  Dynamic binding in a neural network for shape recognition. , 1992, Psychological review.

[9]  T. Poggio,et al.  Are Cortical Models Really Bound by the “Binding Problem”? , 1999, Neuron.

[10]  Edmund T. Rolls,et al.  Invariant Object Recognition in the Visual System with Novel Views of 3D Objects , 2002, Neural Computation.

[11]  I. Biederman,et al.  Effects of illumination intensity and direction on object coding in macaque inferior temporal cortex. , 2002, Cerebral cortex.

[12]  A. Jennekens‐Schinkel Vision, memory and the temporal lobe By Eiichi Iwai and Mortimer Mishkin (eds.), Elsevier, New York, Amsterdam, London, 1990, 453 pages, US$95.00, ISBN 0-444-01531-0 , 1991, Journal of the Neurological Sciences.

[13]  M. Hasselmo,et al.  Object-centered encoding by face-selective neurons in the cortex in the superior temporal sulcus of the monkey , 2004, Experimental Brain Research.

[14]  A. Treves,et al.  The representational capacity of the distributed encoding of information provided by populations of neurons in primate temporal visual cortex , 1997, Experimental Brain Research.

[15]  R. Desimone,et al.  Competitive Mechanisms Subserve Attention in Macaque Areas V2 and V4 , 1999, The Journal of Neuroscience.

[16]  Gustavo Deco,et al.  Large-scale neural model for visual attention: integration of experimental single-cell and fMRI data. , 2002, Cerebral cortex.

[17]  Peter Földiák,et al.  Learning Invariance from Transformation Sequences , 1991, Neural Comput..

[18]  J. Orbach Principles of Neurodynamics. Perceptrons and the Theory of Brain Mechanisms. , 1962 .

[19]  E. Niebur,et al.  Modeling the Temporal Dynamics of IT Neurons in Visual Search: A Mechanism for Top-Down Selective Attention , 1996, Journal of Cognitive Neuroscience.

[20]  T. Poggio,et al.  A network that learns to recognize three-dimensional objects , 1990, Nature.

[21]  G. Deco,et al.  Top-down selective visual attention: A neurodynamical approach , 2001 .

[22]  M. Tovée,et al.  Information encoding in short firing rate epochs by single neurons in the primate temporal visual cortex , 1995 .

[23]  E. Rolls,et al.  Attention and working memory: a dynamical model of neuronal activity in the prefrontal cortex , 2003, The European journal of neuroscience.

[24]  G. Deco,et al.  Cooperation and biased competition model can explain attentional filtering in the prefrontal cortex , 2004, The European journal of neuroscience.

[25]  Roman Bek,et al.  Discourse on one way in which a quantum-mechanics language on the classical logical base can be built up , 1978, Kybernetika.

[26]  A. Cowey,et al.  Topography of the retina and striate cortex and its relationship to visual acuity in rhesus monkeys and squirrel monkeys , 2004, Experimental Brain Research.

[27]  Edmund T Rolls,et al.  Consciousness absent and present: a neurophysiological exploration. , 2004, Progress in brain research.

[28]  E. Rolls,et al.  Object‐based visual neglect: a computational hypothesis , 2002, The European journal of neuroscience.

[29]  R. Desimone,et al.  The Role of Neural Mechanisms of Attention in Solving the Binding Problem , 1999, Neuron.

[30]  Edmund T. Rolls,et al.  Neuronal selectivity, population sparseness, and ergodicity in the inferior temporal visual cortex , 2007, Biological Cybernetics.

[31]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[32]  A. Parker,et al.  Spatial properties of neurons in the monkey striate cortex , 1987, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[33]  E T Rolls,et al.  Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex. , 1995, Journal of neurophysiology.

[34]  Edmund T. Rolls,et al.  Models of invariant object recognition , 2001 .

[35]  E. Adelson,et al.  The analysis of moving visual patterns , 1985 .

[36]  D. Perrett,et al.  Visual neurones responsive to faces in the monkey temporal cortex , 2004, Experimental Brain Research.

[37]  E. Rolls,et al.  Neurodynamics of biased competition and cooperation for attention: a model with spiking neurons. , 2005, Journal of neurophysiology.

[38]  N. Logothetis,et al.  Shape representation in the inferior temporal cortex of monkeys , 1995, Current Biology.

[39]  D. V. van Essen,et al.  A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[40]  Stefano Panzeri,et al.  Information in the Neuronal Representation of Individual Stimuli in the Primate Temporal Visual Cortex , 1997, Journal of Computational Neuroscience.

[41]  E. Rolls,et al.  Size and contrast have only small effects on the responses to faces of neurons in the cortex of the superior temporal sulcus of the monkey , 2004, Experimental Brain Research.

[42]  Edmund T. Rolls,et al.  Position invariant recognition in the visual system with cluttered environments , 2000, Neural Networks.

[43]  E. Rolls The representation of information about faces in the temporal and frontal lobes , 2007, Neuropsychologia.

[44]  E. Rolls,et al.  A Neurodynamical cortical model of visual attention and invariant object recognition , 2004, Vision Research.

[45]  Bartlett W. Mel,et al.  Translation-Invariant Orientation Tuning in Visual “Complex” Cells Could Derive from Intradendritic Computations , 1998, The Journal of Neuroscience.

[46]  Edmund T. Rolls,et al.  Functions of the Primate Temporal Lobe Cortical Visual Areas in Invariant Visual Object and Face Recognition , 2000, Neuron.

[47]  E. Rolls,et al.  Object perception in natural scenes: encoding by inferior temporal cortex simultaneously recorded neurons. , 2005, Journal of neurophysiology.

[48]  E. Rolls Brain mechanisms for invariant visual recognition and learning , 1994, Behavioural Processes.

[49]  Edmund T. Rolls,et al.  Invariant Representations of Objects in Natural Scenes in the Temporal Cortex Visual Areas , 2007 .

[50]  Jan J. Koenderink,et al.  Solid shape , 1990 .

[51]  R A Andersen,et al.  The Analysis of Complex Motion Patterns by Form/Cue Invariant MSTd Neurons , 1996, The Journal of Neuroscience.

[52]  M. Tovée,et al.  Information encoding and the responses of single neurons in the primate temporal visual cortex. , 1993, Journal of neurophysiology.

[53]  E T Rolls,et al.  Neurophysiological mechanisms underlying face processing within and beyond the temporal cortical visual areas. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[54]  Edmund T Rolls,et al.  The Receptive Fields of Inferior Temporal Cortex Neurons in Natural Scenes , 2003, The Journal of Neuroscience.

[55]  Thomas P. Trappenberg,et al.  Effective Size of Receptive Fields of Inferior Temporal Visual Cortex Neurons in Natural Scenes , 2001, NIPS.

[56]  G. Deco,et al.  The time course of selective visual attention: theory and experiments , 2002, Vision Research.

[57]  C. Malsburg Self-organization of orientation sensitive cells in the striate cortex , 2004, Kybernetik.

[58]  Bartlett W. Mel,et al.  Minimizing Binding Errors Using Learned Conjunctive Features , 2000, Neural Computation.

[59]  Charles G. Gross,et al.  Pattern recognition mechanisms , 1985 .

[60]  J. Koenderink,et al.  The internal representation of solid shape with respect to vision , 1979, Biological Cybernetics.

[61]  Bruno A. Olshausen,et al.  A multiscale dynamic routing circuit for forming size- and position-invariant object representations , 1995, Journal of Computational Neuroscience.

[62]  Keiji Tanaka,et al.  Inferotemporal cortex and object vision. , 1996, Annual review of neuroscience.

[63]  M. Tovée,et al.  The responses of neurons in the temporal cortex of primates, and face identification and detection , 1994, Experimental Brain Research.

[64]  P König,et al.  Formation of cortical cell assemblies. , 1990, Cold Spring Harbor symposia on quantitative biology.

[65]  John Duncan,et al.  A neural basis for visual search in inferior temporal cortex , 1993, Nature.

[66]  Edmund T. Rolls,et al.  Learning invariant object recognition in the visual system with continuous transformations , 2006, Biological Cybernetics.

[67]  M. Hasselmo,et al.  The role of expression and identity in the face-selective responses of neurons in the temporal visual cortex of the monkey , 1989, Behavioural Brain Research.

[68]  E. Rolls,et al.  INVARIANT FACE AND OBJECT RECOGNITION IN THE VISUAL SYSTEM , 1997, Progress in Neurobiology.

[69]  A. Cowey,et al.  Human cortical magnification factor and its relation to visual acuity , 2004, Experimental Brain Research.

[70]  Gustavo Deco,et al.  Computational neuroscience of vision , 2002 .

[71]  Edmund T. Rolls,et al.  Invariant recognition of feature combinations in the visual system , 2002, Biological Cybernetics.

[72]  Roland Baddeley,et al.  Optimal, Unsupervised Learning in Invariant Object Recognition , 1997, Neural Computation.

[73]  Eric R. Kandel,et al.  Perception of motion, depth and form , 2000 .

[74]  S. Yamane,et al.  What facial features activate face neurons in the inferotemporal cortex of the monkey? , 2004, Experimental Brain Research.

[75]  E. Rolls,et al.  Attention, short-term memory, and action selection: A unifying theory , 2005, Progress in Neurobiology.

[76]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[77]  E. Rolls,et al.  Information in the first spike, the order of spikes, and the number of spikes provided by neurons in the inferior temporal visual cortex , 2006, Vision Research.

[78]  E. Rolls,et al.  Decision‐making and Weber's law: a neurophysiological model , 2006, The European journal of neuroscience.

[79]  E. Rolls,et al.  Scene perception: inferior temporal cortex neurons encode the positions of different objects in the scene , 2005, The European journal of neuroscience.

[80]  Mark H. Johnson,et al.  Object Recognition and Sensitive Periods: A Computational Analysis of Visual Imprinting , 1994, Neural Computation.

[81]  N. Logothetis,et al.  View-dependent object recognition by monkeys , 1994, Current Biology.

[82]  Gustavo Deco,et al.  Feature-based Attention in Human Visual Cortex: Simulation of Fmri Data , 2003 .

[83]  A. J. Mistlin,et al.  Visual cells in the temporal cortex sensitive to face view and gaze direction , 1985, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[84]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[85]  Edmund T. Rolls,et al.  A Model of Invariant Object Recognition in the Visual System: Learning Rules, Activation Functions, Lateral Inhibition, and Information-Based Performance Measures , 2000, Neural Computation.

[86]  J. Rothwell Principles of Neural Science , 1982 .

[87]  W Singer,et al.  Visual feature integration and the temporal correlation hypothesis. , 1995, Annual review of neuroscience.

[88]  Michael C. Mozer,et al.  Perception of multiple objects - a connectionist approach , 1991, Neural network modeling and connectionism.

[89]  R. Desimone,et al.  Selective attention gates visual processing in the extrastriate cortex. , 1985, Science.

[90]  M. Tovée,et al.  Translation invariance in the responses to faces of single neurons in the temporal visual cortical areas of the alert macaque. , 1994, Journal of neurophysiology.

[91]  Edmund T. Rolls,et al.  Invariant Global Motion Recognition in the Dorsal Visual System: A Unifying Theory , 2007, Neural Computation.

[92]  S. Treue,et al.  Attentional Modulation Strength in Cortical Area MT Depends on Stimulus Contrast , 2002, Neuron.

[93]  James L. McGaugh,et al.  Brain Organization and Memory: Cells, Systems, and Circuits , 1992 .

[94]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[95]  Gustavo Deco,et al.  Attention in natural scenes: Neurophysiological and computational bases , 2006, Neural Networks.

[96]  E. Rolls,et al.  The Neurophysiology of Backward Visual Masking: Information Analysis , 1999, Journal of Cognitive Neuroscience.

[97]  David I. Perrett,et al.  Neurophysiology of shape processing , 1993, Image Vis. Comput..

[98]  E. Rolls,et al.  View-invariant representations of familiar objects by neurons in the inferior temporal visual cortex. , 1998, Cerebral cortex.

[99]  Leonardo Franco,et al.  The use of decoding to analyze the contribution to the information of the correlations between the firing of simultaneously recorded neurons , 2004, Experimental Brain Research.

[100]  E. Rolls,et al.  Neural networks and brain function , 1998 .

[101]  E. Rolls,et al.  Synaptic and spiking dynamics underlying reward reversal in the orbitofrontal cortex. , 2004, Cerebral cortex.

[102]  G. Edelman,et al.  Spatial signaling in the development and function of neural connections. , 1991, Cerebral cortex.

[103]  Edmund T. Rolls,et al.  Information encoding in the inferior temporal visual cortex: contributions of the firing rates and the correlations between the firing of neurons , 2010, Biological Cybernetics.

[104]  S. Ullman High-Level Vision: Object Recognition and Visual Cognition , 1996 .

[105]  E. Oja Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[106]  E. Rolls,et al.  Spatial vs temporal continuity in view invariant visual object recognition learning , 2006, Vision Research.

[107]  J. Feldman Four frames suffice: A provisional model of vision and space , 1985, Behavioral and Brain Sciences.

[108]  E. Rolls,et al.  What and Where in Visual Working Memory: A Computational Neurodynamical Perspective for Integrating fMRI and Single-Neuron Data , 2004, Journal of Cognitive Neuroscience.

[109]  A G Barto,et al.  Toward a modern theory of adaptive networks: expectation and prediction. , 1981, Psychological review.

[110]  Tai Sing Lee,et al.  A unified model of spatial and object attention based on inter-cortical biased competition , 2002, Neurocomputing.

[111]  K. H. Britten,et al.  Neuronal correlates of a perceptual decision , 1989, Nature.

[112]  Tomaso Poggio,et al.  Models of object recognition , 2000, Nature Neuroscience.

[113]  E. Rolls Learning mechanisms in the temporal lobe visual cortex , 1995, Behavioural Brain Research.

[114]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[115]  E T Rolls,et al.  Invariant object recognition in the visual system with error correction and temporal difference learning , 2001, Network.