A neural code for three-dimensional object shape in macaque inferotemporal cortex

Previous investigations of the neural code for complex object shape have focused on two-dimensional pattern representation. This may be the primary mode for object vision given its simplicity and direct relation to the retinal image. In contrast, three-dimensional shape representation requires higher-dimensional coding derived from extensive computation. We found evidence for an explicit neural code for complex three-dimensional object shape. We used an evolutionary stimulus strategy and linear/nonlinear response models to characterize three-dimensional shape responses in macaque monkey inferotemporal cortex (IT). We found widespread tuning for three-dimensional spatial configurations of surface fragments characterized by their three-dimensional orientations and joint principal curvatures. Configural representation of three-dimensional shape could provide specific knowledge of object structure to support guidance of complex physical interactions and evaluation of object functionality and utility.

[1]  D. B. Bender,et al.  Visual properties of neurons in inferotemporal cortex of the Macaque. , 1972, Journal of neurophysiology.

[2]  D. Marr,et al.  Representation and recognition of the spatial organization of three-dimensional shapes , 1978, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[3]  R. Mansfield,et al.  Analysis of visual behavior , 1982 .

[4]  R. Desimone,et al.  Shape recognition and inferior temporal neurons. , 1983, Proceedings of the National Academy of Sciences of the United States of America.

[5]  Donald D. Hoffman,et al.  Parts of recognition , 1984, Cognition.

[6]  J J Koenderink,et al.  What Does the Occluding Contour Tell Us about Solid Shape? , 1984, Perception.

[7]  I. Biederman.  Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[8]  M. Tarr,et al.  Mental rotation and orientation-dependence in shape recognition , 1989, Cognitive Psychology.

[9]  Edmund T. Rolls,et al.  The relative advantages of sparse versus distributed encoding for associative neuronal networks in the brain , 1990 .

[10]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[11]  Minami Ito,et al.  Columns for visual features of objects in monkey inferotemporal cortex , 1992, Nature.

[12]  D. V. van Essen,et al.  Selectivity for polar, hyperbolic, and Cartesian gratings in macaque visual cortex. , 1993, Science.

[13]  Keiji Tanaka,et al.  Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex. , 1994, Journal of neurophysiology.

[14]  H H Bülthoff,et al.  How are three-dimensional objects represented in the brain? , 1994, Cerebral cortex.

[15]  Minami Ito,et al.  Size and position invariance of neuronal responses in monkey inferotemporal cortex. , 1995, Journal of neurophysiology.

[16]  T Poggio,et al.  View-based models of 3D object recognition: invariance to imaging transformations. , 1995, Cerebral cortex.

[17]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[18]  K Tsutsui,et al.  Neural coding of 3D features of objects for hand action in the parietal cortex of the monkey. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[19]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[20]  C. Connor,et al.  Responses to contour features in macaque area V4. , 1999, Journal of neurophysiology.

[21]  R Vogels,et al.  Macaque inferior temporal neurons are selective for disparity-defined three-dimensional shapes. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[22]  G. Orban,et al.  Selectivity for 3D shape that reveals distinct areas within macaque inferior temporal cortex. , 2000, Science.

[23]  J L Gallant,et al.  Sparse coding and decorrelation in primary visual cortex during natural vision. , 2000, Science.

[24]  I. Fujita,et al.  Disparity selectivity of neurons in monkey inferior temporal cortex. , 2000, Journal of neurophysiology.

[25]  Tomaso Poggio,et al.  Models of object recognition , 2000, Nature Neuroscience.

[26]  G. Orban,et al.  Three-Dimensional Shape Coding in Inferior Temporal Cortex , 2000, Neuron.

[27]  C. Connor,et al.  Shape representation in area V4: position-specific tuning for boundary conformation. , 2001, Journal of neurophysiology.

[28]  Z Kourtzi,et al.  Representation of Perceived Object Shape by the Human Lateral Occipital Complex , 2001, Science.

[29]  D. Wilkin,et al.  Neuron , 2001, Brain Research.

[30]  David J. Freedman,et al.  Categorical representation of visual stimuli in the primate prefrontal cortex. , 2001, Science.

[31]  Y. Yamane,et al.  Complex objects are represented in macaque inferotemporal cortex by the combination of feature columns , 2001, Nature Neuroscience.

[32]  C. Connor,et al.  Population coding of shape in area V4 , 2002, Nature Neuroscience.

[33]  Ichiro Fujita,et al.  Disparity-selective neurons in area V4 of macaque monkeys. , 2002, Journal of neurophysiology.

[34]  C. Connor,et al.  Three-dimensional orientation tuning in macaque area V4 , 2002, Nature Neuroscience.

[35]  M. Behrmann,et al.  Impact of learning on representation of parts and wholes in monkey inferotemporal cortex , 2002, Nature Neuroscience.

[36]  Doris Y. Tsao,et al.  Faces and objects in macaque cerebral cortex , 2003, Nature Neuroscience.

[37]  Minami Ito,et al.  Representation of Angles Embedded within Contour Stimuli in Area V2 of Macaque Monkeys , 2004, The Journal of Neuroscience.

[38]  Charles E Connor,et al.  Underlying principles of visual shape selectivity in posterior inferotemporal cortex , 2004, Nature Neuroscience.

[39]  C. Koch,et al.  Invariant visual representation by single neurons in the human brain , 2005, Nature.

[40]  Keiji Tanaka,et al.  Prior experience of rotation is not required for recognizing objects seen from different angles , 2005, Nature Neuroscience.

[41]  P. Roelfsema.  Cortical algorithms for perceptual grouping. , 2006, Annual review of neuroscience.

[42]  Eero P. Simoncelli,et al.  How MT cells analyze the motion of visual patterns , 2006, Nature Neuroscience.

[43]  Scott L. Brincat,et al.  Dynamic Shape Synthesis in Posterior Inferotemporal Cortex , 2006, Neuron.

[44]  Michael J. Tarr,et al.  RECONSIDERING THE ROLE OF STRUCTURE IN VISION , 2006 .

[45]  Thomas Serre,et al.  A feedforward architecture accounts for rapid categorization , 2007, Proceedings of the National Academy of Sciences.

[46]  Anitha Pasupathy,et al.  Transformation of shape information in the ventral pathway , 2007, Current Opinion in Neurobiology.

[47]  D. C. Essen,et al.  Neurons in monkey visual area V2 encode combinations of orientations , 2007, Nature Neuroscience.