Three-Dimensional Shape: Cortical Mechanisms of Shape Extraction

Three-dimensional (3D) shape is perceived by human and nonhuman primates from a variety of cues including stereo, motion, texture, and shading. Functional imaging in awake monkeys and humans concur in the involvement of MT/V5 and its human homolog in the extraction of 3D shape form motion, along with a number of parietal and occipitotemporal regions. Single-cell studies have highlighted the selectivity for gradients in speed, disparity, and texture and demonstrated cue convergence in MT/V5, but also in the parietal caudal intraparietal (CIP) region and the inferotemporal (IT) TEs region. While most neurons in these regions were documented to be selective for linear gradients, those in TEs are selective for second-order gradients of disparity. Thus 3D shape is not represented as a 3D map but more likely by sets of local gradients.

[1]  H. Bülthoff,et al.  Representation of the perceived 3-D object shape in the human lateral occipital complex. , 2003, Cerebral cortex.

[2]  A. Parker.  Binocular depth perception and the cerebral cortex , 2007, Nature Reviews Neuroscience.

[3]  T. Poggio,et al.  A computational theory of human stereo vision , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[4]  Guy A. Orban,et al.  Mapping the parietal cortex of human and non-human primates , 2006, Neuropsychologia.

[5]  T. Hendler,et al.  Visuo-haptic object-related activation in the ventral visual pathway , 2001, Nature Neuroscience.

[6]  D. Heeger,et al.  Retinotopy and Functional Subdivision of Human Areas MT and MST , 2002, The Journal of Neuroscience.

[7]  Karl J. Friston,et al.  A direct demonstration of functional specialization in human visual cortex , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[8]  Peter Janssen,et al.  Anterior Regions of Monkey Parietal Cortex Process Visual 3D Shape , 2007, Neuron.

[9]  H. Sakata,et al.  Integration of perspective and disparity cues in surface-orientation-selective neurons of area CIP. , 2001, Journal of neurophysiology.

[10]  Keiji Tanaka,et al.  Connections between Anterior Inferotemporal Cortex and Superior Temporal Sulcus Regions in the Macaque Monkey , 2000, The Journal of Neuroscience.

[11]  R Vogels,et al.  Macaque inferior temporal neurons are selective for disparity-defined three-dimensional shapes. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[12]  B. Rogers,et al.  Disparity curvature and the perception of three-dimensional surfaces , 1989, Nature.

[13]  D. B. Bender,et al.  Visual properties of neurons in inferotemporal cortex of the Macaque. , 1972, Journal of neurophysiology.

[14]  R. Wurtz,et al.  Sensitivity of MST neurons to optic flow stimuli. I. A continuum of response selectivity to large-field stimuli. , 1991, Journal of neurophysiology.

[15]  G. Orban,et al.  Attention to Speed of Motion, Speed Discrimination, and Task Difficulty: An fMRI Study , 2000, NeuroImage.

[16]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[17]  Keiji Tanaka,et al.  Integration of direction signals of image motion in the superior temporal sulcus of the macaque monkey , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[18]  Ichiro Fujita,et al.  Disparity-selective neurons in area V4 of macaque monkeys. , 2002 .

[19]  R J Zatorre,et al.  Stereopsis after unilateral anterior temporal lobectomy. Dissociation between local and global measures. , 1991, Brain : a journal of neurology.

[20]  G. Orban,et al.  Selectivity for 3D shape that reveals distinct areas within macaque inferior temporal cortex. , 2000, Science.

[21]  Ichiro Fujita,et al.  Neural Correlates of Fine Depth Discrimination in Monkey Inferior Temporal Cortex , 2005, The Journal of Neuroscience.

[22]  Keiji Tanaka,et al.  Coding visual images of objects in the inferotemporal cortex of the macaque monkey. , 1991, Journal of neurophysiology.

[23]  A. Parker,et al.  Perceptually Bistable Three-Dimensional Figures Evoke High Choice Probabilities in Cortical Area MT , 2001, The Journal of Neuroscience.

[24]  R. Vogels.  Categorization of complex visual images by rhesus monkeys. Part 2: single‐cell study , 1999, The European journal of neuroscience.

[25]  R. Andersen,et al.  Center–Surround Antagonism Based on Disparity in Primate Area MT , 1998, The Journal of Neuroscience.

[26]  Keiji Tanaka,et al.  Inferotemporal cortex and object vision. , 1996, Annual review of neuroscience.

[27]  Jay Hegdé,et al.  Stimulus dependence of disparity coding in primate visual area V4. , 2005, Journal of neurophysiology.

[28]  J. Todd.  Review TRENDS in Cognitive Sciences Vol.8 No.3 March 2004 The visual perception of 3D shape q , 2022 .

[29]  G. Orban,et al.  Spatial heterogeneity of inhibitory surrounds in the middle temporal visual area. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[30]  G. Orban,et al.  Cue-invariant shape selectivity of macaque inferior temporal neurons. , 1993, Science.

[31]  Gregory C DeAngelis,et al.  Coding of horizontal disparity and velocity by MT neurons in the alert macaque. , 2003, Journal of neurophysiology.

[32]  Guy A. Orban,et al.  Visual Processing in Macaque Area MT/V5 and Its Satellites (MSTd and MSTv) , 1997 .

[33]  Tomoka Naganuma,et al.  Neural Correlates for Perception of 3D Surface Orientation from Texture Gradient , 2002, Science.

[34]  C. Marzi,et al.  Visual pattern and form perception in the rhesus monkey following ablation of the superior colliculus , 1977, Brain Research.

[35]  D. V. van Essen,et al.  Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey , 2000, The Journal of comparative neurology.

[36]  M. Braunstein.  Perceived direction of rotation of simulated three-dimensional patterns , 1977 .

[37]  G. Orban,et al.  The spatial distribution of the antagonistic surround of MT/V5 neurons. , 1997, Cerebral cortex.

[38]  R. M. Siegel,et al.  Three-dimensional structure-from-motion selectivity in the anterior superior temporal polysensory area, STPa, of the behaving monkey. , 2005, Cerebral cortex.

[39]  J T Todd,et al.  Perception of local three-dimensional shape. , 1996, Journal of experimental psychology. Human perception and performance.

[40]  David J. Fleet,et al.  Human cortical activity correlates with stereoscopic depth perception. , 2001, Journal of neurophysiology.

[41]  R A Andersen,et al.  Neural responses to velocity gradients in macaque cortical area MT , 1996, Visual Neuroscience.

[42]  R. Desimone,et al.  Shape recognition and inferior temporal neurons. , 1983, Proceedings of the National Academy of Sciences of the United States of America.

[43]  C. Connor,et al.  Three-dimensional orientation tuning in macaque area V4 , 2002, Nature Neuroscience.

[44]  G. Orban,et al.  Visual Motion Processing Investigated Using Contrast Agent-Enhanced fMRI in Awake Behaving Monkeys , 2001, Neuron.

[45]  Mark F. Bradshaw,et al.  Sensitivity to horizontal and vertical corrugations defined by binocular disparity , 1999, Vision Research.

[46]  B Rogers,et al.  Motion Parallax as an Independent Cue for Depth Perception , 1979, Perception.

[47]  Minami Ito,et al.  Size and position invariance of neuronal responses in monkey inferotemporal cortex. , 1995, Journal of neurophysiology.

[48]  James M. Hillis,et al.  Slant from texture and disparity cues: optimal cue combination. , 2004, Journal of vision.

[49]  H. Sakata,et al.  Parietal neurons represent surface orientation from the gradient of binocular disparity. , 2000, Journal of neurophysiology.

[50]  D. V. van Essen,et al.  The Processing of Visual Shape in the Cerebral Cortex of Human and Nonhuman Primates: A Functional Magnetic Resonance Imaging Study , 2004, The Journal of Neuroscience.

[51]  Joseph S. Gati,et al.  Differences in perceived shape from shading correlate with activity in early visual areas , 1997, Current Biology.

[52]  H. Sakata,et al.  From Three-Dimensional Space Vision to Prehensile Hand Movements: The Lateral Intraparietal Area Links the Area V3A and the Anterior Intraparietal Area in Macaques , 2001, The Journal of Neuroscience.

[53]  C. Gross,et al.  Effects of foveal prestriate and inferotemporal lesions on visual discrimination by rhesus monkeys , 2004, Experimental Brain Research.

[54]  James T Todd,et al.  The visual perception of 3-D shape from multiple cues: Are observers capable of perceiving metric structure? , 2003, Perception & psychophysics.

[55]  R. Andersen,et al.  Response of MSTd neurons to simulated 3D orientation of rotating planes. , 2002, Journal of neurophysiology.

[56]  Bettina Sorger,et al.  Human Cortical Object Recognition from a Visual Motion Flowfield , 2003, The Journal of Neuroscience.

[57]  Ian P. Howard,et al.  Binocular Vision and Stereopsis , 1996 .

[58]  J. Bakin,et al.  Visual Responses in Monkey Areas V1 and V2 to Three-Dimensional Surface Configurations , 2000, The Journal of Neuroscience.

[59]  D. Bradley,et al.  Neural population code for fine perceptual decisions in area MT , 2005, Nature Neuroscience.

[60]  James T. Todd,et al.  The effects of field of view on the perception of 3D slant from texture , 2005, Vision Research.

[61]  Y. Miyashita,et al.  Functional MRI of Macaque Monkeys Performing a Cognitive Set-Shifting Task , 2002, Science.

[62]  Ravi S. Menon,et al.  Haptic study of three-dimensional objects activates extrastriate visual areas , 2002, Neuropsychologia.

[63]  Marc M. Van Hulle,et al.  Function of center-surround antagonism for motion in visual area MT/V5: a modeling study , 2001, Vision Research.

[64]  Leslie G. Ungerleider,et al.  Cortical connections of visual area MT in the macaque , 1986, The Journal of comparative neurology.

[65]  Hideko F. Norman,et al.  Visual discrimination of local surface structure: Slant, tilt, and curvedness , 2006, Vision Research.

[66]  G A Orban,et al.  Orientation discrimination of objects and gratings compared: an fMRI study , 2001, The European journal of neuroscience.

[67]  Jerry D. Nguyenkim,et al.  Disparity-Based Coding of Three-Dimensional Surface Orientation by Macaque Middle Temporal Neurons , 2003, The Journal of Neuroscience.

[68]  Astrid M L Kappers,et al.  Perception of Surface Reflectance of 3-D Geometrical Shapes: Influence of the Lighting Mode , 2003, Perception.

[69]  Guy A Orban,et al.  Higher order visual processing in macaque extrastriate cortex. , 2008, Physiological reviews.

[70]  R. Wurtz,et al.  Medial Superior Temporal Area Neurons Respond to Speed Patterns in Optic Flow , 1997, The Journal of Neuroscience.

[71]  H. Sakata,et al.  Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP. , 2000, Journal of neurophysiology.

[72]  D C Van Essen,et al.  Information processing in the primate visual system: an integrated systems perspective. , 1992, Science.

[73]  John H. R. Maunsell,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. II. Binocular interactions and sensitivity to binocular disparity. , 1983, Journal of neurophysiology.

[74]  G. Orban,et al.  Attention to 3-D Shape, 3-D Motion, and Texture in 3-D Structure from Motion Displays , 2004, Journal of Cognitive Neuroscience.

[75]  A. Parker,et al.  Receptive Field Size in V1 Neurons Limits Acuity for Perceiving Disparity Modulation , 2004, The Journal of Neuroscience.

[76]  G. Orban,et al.  Three-Dimensional Shape Coding in Inferior Temporal Cortex , 2000, Neuron.

[77]  T. Chaminade,et al.  The evolutionary neuroscience of tool making , 2007, Neuropsychologia.

[78]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[79]  R. Andersen,et al.  Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[80]  Scott O. Murray,et al.  Processing Shape, Motion and Three-dimensional Shape-from-motion in the Human Cortex , 2003 .

[81]  G. Orban,et al.  Motion-responsive regions of the human brain , 1999, Experimental Brain Research.

[82]  S. Edelman,et al.  Orientation dependence in the recognition of familiar and novel views of three-dimensional objects , 1992, Vision Research.

[83]  H. Sakata,et al.  Selectivity of the parietal visual neurones in 3D orientation of surface of stereoscopic stimuli. , 1996, Neuroreport.

[84]  A Berthoz,et al.  Visual perception of motion and 3-D structure from motion: an fMRI study. , 2000, Cerebral cortex.

[85]  Astrid M L Kappers,et al.  The Perception of Doubly Curved Surfaces From Anisotropic Textures , 2004, Psychological science.

[86]  K Tsutsui,et al.  Neural coding of 3D features of objects for hand action in the parietal cortex of the monkey. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[87]  I. Fujita,et al.  Disparity selectivity of neurons in monkey inferior temporal cortex. , 2000, Journal of neurophysiology.

[88]  G. Orban,et al.  Charting the Lower Superior Temporal Region, a New Motion-Sensitive Region in Monkey Superior Temporal Sulcus , 2006, The Journal of Neuroscience.

[89]  J T Todd,et al.  Ambiguity and the ‘Mental Eye’ in Pictorial Relief , 2001, Perception.

[90]  G. Orban,et al.  Selectivity of Neuronal Adaptation Does Not Match Response Selectivity: A Single-Cell Study of the fMRI Adaptation Paradigm , 2006, Neuron.

[91]  H. Komatsu,et al.  Influence of the Direction of Elemental Luminance Gradients on the Responses of V4 Cells to Textured Surfaces , 2001, The Journal of Neuroscience.

[92]  G. Orban,et al.  Color discrimination involves ventral and dorsal stream visual areas. , 2004, Cerebral cortex.

[93]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[94]  G. Orban,et al.  Responses of macaque STS neurons to optic flow components: a comparison of areas MT and MST. , 1994, Journal of neurophysiology.

[95]  G. Orban,et al.  Selectivity of Macaque MT/V5 Neurons for Surface Orientation in Depth Specified by Motion , 1997, The European journal of neuroscience.

[96]  Guy A. Orban,et al.  Similarities and differences in motion processing between the human and macaque brain: evidence from fMRI , 2003, Neuropsychologia.

[97]  H. Bülthoff,et al.  3D shape perception from combined depth cues in human visual cortex , 2005, Nature Neuroscience.

[98]  R Vogels,et al.  Coding of stimulus invariances by inferior temporal neurons. , 1996, Progress in brain research.

[99]  J F Norman,et al.  The visual perception of rigid motion from constant flow fields , 1996, Perception & psychophysics.

[100]  Jay Hegdé,et al.  Role of primate visual area V4 in the processing of 3-D shape characteristics defined by disparity. , 2005, Journal of neurophysiology.

[101]  G. Orban,et al.  Comparative mapping of higher visual areas in monkeys and humans , 2004, Trends in Cognitive Sciences.

[102]  P. Cavanagh,et al.  Retinotopy and color sensitivity in human visual cortical area V8 , 1998, Nature Neuroscience.

[103]  Guy A. Orban,et al.  The neuronal machinery involved in successive orientation discrimination , 1998, Progress in Neurobiology.

[104]  M. Graziano,et al.  Tuning of MST neurons to spiral motions , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[105]  Takahiro Doi,et al.  Disparity-tuning characteristics of neuronal responses to dynamic random-dot stereograms in macaque visual area V4. , 2005, Journal of neurophysiology.

[106]  B. G. Cumming,et al.  Responses of primary visual cortical neurons to binocular disparity without depth perception , 1997, Nature.

[107]  D C Van Essen,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. , 1983, Journal of neurophysiology.

[108]  I. Fujita,et al.  Rejection of False Matches for Binocular Correspondence in Macaque Visual Cortical Area V4 , 2004, The Journal of Neuroscience.

[109]  C E Connor,et al.  Disparity tuning in macaque area V4 , 2001, Neuroreport.

[110]  R. J. Seitz,et al.  A fronto‐parietal circuit for object manipulation in man: evidence from an fMRI‐study , 1999, The European journal of neuroscience.

[111]  G Westheimer,et al.  The spatial sense of the eye. Proctor lecture. , 1979, Investigative ophthalmology & visual science.

[112]  G. Orban,et al.  Perception of Three-Dimensional Shape From Specular Highlights, Deformations of Shading, and Other Types of Visual Information , 2004, Psychological science.

[113]  M. Taira,et al.  Cortical Areas Related to Attention to 3D Surface Structures Based on Shading: An fMRI Study , 2001, NeuroImage.

[114]  Guy A. Orban,et al.  Functional neuroanatomy for the processing of 3D shape from shading and texture in humans , 2005 .

[115]  S. Zeki,et al.  The architecture of the colour centre in the human visual brain: new results and a review * , 2000, The European journal of neuroscience.

[116]  R. M. Siegel,et al.  Perception of three-dimensional structure from motion in monkey and man , 1988, Nature.

[117]  R. Andersen,et al.  Encoding of three-dimensional structure-from-motion by primate area MT neurons , 1998, Nature.

[118]  Doris Y. Tsao,et al.  A Cortical Region Consisting Entirely of Face-Selective Cells , 2006, Science.

[119]  G. Orban,et al.  Speed and direction selectivity of macaque middle temporal neurons. , 1993, Journal of neurophysiology.

[120]  Peter Janssen,et al.  Extracting 3D structure from disparity , 2006, Trends in Neurosciences.

[121]  K V Shenoy,et al.  Neural mechanisms for heading and structure-from-motion perception. , 1996, Cold Spring Harbor symposia on quantitative biology.

[122]  G. Poggio,et al.  Binocular interaction and depth sensitivity in striate and prestriate cortex of behaving rhesus monkey. , 1977, Journal of neurophysiology.

[123]  Guy Orban,et al.  Cortical regions involved in extracting 3D shape from shading: A human and monkey fMRI study , 2003 .

[124]  Rufin Vogels,et al.  Convergence of Depth from Texture and Depth from Disparity in Macaque Inferior Temporal Cortex , 2004, The Journal of Neuroscience.

[125]  Doris Y. Tsao,et al.  Stereopsis Activates V3A and Caudal Intraparietal Areas in Macaques and Humans , 2003, Neuron.

[126]  H. Sakata,et al.  Toward an understanding of the neural processing for 3D shape perception , 2005, Neuropsychologia.

[127]  A. Parker,et al.  Comparing perceptual signals of single V5/MT neurons in two binocular depth tasks. , 2004, Journal of neurophysiology.

[128]  J. Todd,et al.  Effects of Texture, Illumination, and Surface Reflectance on Stereoscopic Shape Perception , 1996, Perception.

[129]  D. Bradley,et al.  Structure and function of visual area MT. , 2005, Annual review of neuroscience.

[130]  David J. Kriegman,et al.  The Bas-Relief Ambiguity , 2004, International Journal of Computer Vision.

[131]  G. DeAngelis,et al.  Contribution of Middle Temporal Area to Coarse Depth Discrimination: Comparison of Neuronal and Psychophysical Sensitivity , 2003, The Journal of Neuroscience.

[132]  H. Wallach,et al.  The kinetic depth effect. , 1953, Journal of experimental psychology.

[133]  Simon J Watt,et al.  The visual control of reaching and grasping: binocular disparity and motion parallax. , 2003, Journal of experimental psychology. Human perception and performance.

[134]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[135]  J. Gibson.  The perception of visual surfaces. , 1950, The American journal of psychology.

[136]  Guy Marchal,et al.  Human Cortical Regions Involved in Extracting Depth from Motion , 1999, Neuron.

[137]  G. Orban,et al.  How well do response changes of striate neurons signal differences in orientation: a study in the discriminating monkey , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[138]  Nicholas J. Priebe,et al.  The Neural Representation of Speed in Macaque Area MT/V5 , 2003, The Journal of Neuroscience.

[139]  T. Albright,et al.  Contribution of area MT to perception of three-dimensional shape: a computational study , 1996, Vision Research.

[140]  J. Movshon,et al.  The analysis of visual motion: a comparison of neuronal and psychophysical performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[141]  R. M. Siegel,et al.  The Perception of Structure from Visual Motion in Monkey and Man , 1990, Journal of Cognitive Neuroscience.

[142]  James M. Hillis,et al.  Combining Sensory Information: Mandatory Fusion Within, but Not Between, Senses , 2002, Science.

[143]  G Sperling,et al.  How to study the kinetic depth effect experimentally. , 1990, Journal of experimental psychology. Human perception and performance.

[144]  G. Orban,et al.  Extracting 3D from Motion: Differences in Human and Monkey Intraparietal Cortex , 2002, Science.

[145]  N. Kanwisher,et al.  Cortical Regions Involved in Perceiving Object Shape , 2000, The Journal of Neuroscience.

[146]  Leslie G. Ungerleider.  Two cortical visual systems , 1982 .

[147]  E. DeYoe,et al.  Mapping striate and extrastriate visual areas in human cerebral cortex. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[148]  H. Sakata,et al.  The TINS Lecture The parietal association cortex in depth perception and visual control of hand action , 1997, Trends in Neurosciences.

[149]  Peter Janssen,et al.  Higher order disparity sensitive regions in human cortex? an fMRI study , 2002 .

[150]  J. Koenderink,et al.  Light Direction from Shad(ow)ed Random Gaussian Surfaces , 2004, Perception.

[151]  C. Büchel,et al.  Surface orientation discrimination activates caudal and anterior intraparietal sulcus in humans: an event-related fMRI study. , 2001, Journal of neurophysiology.

[152]  Nikos K. Logothetis,et al.  Three-Dimensional Shape Representation in Monkey Cortex , 2002, Neuron.

[153]  V. Mountcastle,et al.  Capacities of humans and monkeys to discriminate vibratory stimuli of different frequency and amplitude: a correlation between neural events and psychological measurements. , 1975, Journal of neurophysiology.

[154]  G. Orban,et al.  Macaque Inferior Temporal Neurons Are Selective for Three-Dimensional Boundaries and Surfaces , 2001, The Journal of Neuroscience.

[155]  G. Orban,et al.  At Least at the Level of Inferior Temporal Cortex, the Stereo Correspondence Problem Is Solved , 2003, Neuron.