The Processing of Three-Dimensional Shape from Disparity in the Human Brain

Three-dimensional (3D) shape is important for the visual control of grasping and manipulation and for object recognition. Although there has been some progress in our understanding of how 3D shape is extracted from motion and other monocular cues, little is known of how the human brain extracts 3D shape from disparity, commonly regarded as the strongest depth cue. Previous fMRI studies in the awake monkey have established that the interaction between stereo (present or absent) and the order of disparity (zero or second order) constitutes the MR signature of regions housing second-order disparity-selective neurons (Janssen et al., 2000; Srivastava et al., 2006; Durand et al., 2007; Joly et al., 2007). Testing the interaction between stereo and order of disparity in a large cohort of human subjects, revealed the involvement of five IPS regions (VIPS/V7*, POIPS, DIPSM, DIPSA, and phAIP), as well as V3 and the V3A complex in occipital cortex, the posterior inferior temporal gyrus (ITG), and ventral premotor cortex (vPrCS) in the extraction and processing of 3D shape from stereo. Control experiments ruled out attention and convergence eye movements as confounding factors. Many of these regions, DIPSM, DIPSA, phAIP, and probably posterior ITG and ventral premotor cortex, correspond to monkey regions with similar functionality, whereas the evolutionarily new or modified regions are located in occipital (the V3A complex) and occipitoparietal cortex (VIPS/V7* and POIPS). Interestingly, activity in these occipital regions correlates with the depth amplitude perceived by the subjects in the 3D surfaces used as stimuli in these fMRI experiments.

[1]  G. Poggio,et al.  Stereoscopic mechanisms in monkey visual cortex: binocular correlation and disparity selectivity , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[2]  J. Frahm,et al.  Direct FLASH MR imaging of magnetic field inhomogeneities by gradient compensation , 1988, Magnetic resonance in medicine.

[3]  J W Belliveau,et al.  Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. , 1995, Science.

[4]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[5]  A. Dale,et al.  Functional Analysis of V3A and Related Areas in Human Visual Cortex , 1997, The Journal of Neuroscience.

[6]  M. Raichle,et al.  Anatomic Localization and Quantitative Analysis of Gradient Refocused Echo-Planar fMRI Susceptibility Artifacts , 1997, NeuroImage.

[7]  A. Dale,et al.  The Retinotopy of Visual Spatial Attention , 1998, Neuron.

[8]  Makoto Kato,et al.  Processing of shape defined by disparity in monkey inferior temporal cortex , 1998, Neuroscience Research.

[9]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[10]  R. J. Seitz,et al.  A parieto-premotor network for object manipulation: evidence from neuroimaging , 1999, Experimental Brain Research.

[11]  G. Orban,et al.  Motion-responsive regions of the human brain , 1999, Experimental Brain Research.

[12]  R. J. Seitz,et al.  A fronto‐parietal circuit for object manipulation in man: evidence from an fMRI‐study , 1999, The European journal of neuroscience.

[13]  R Vogels,et al.  Macaque inferior temporal neurons are selective for disparity-defined three-dimensional shapes. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[14]  A. Dale,et al.  Cortical Surface-Based Analysis II: Inflation, Flattening, and a Surface-Based Coordinate System , 1999, NeuroImage.

[15]  Guy Marchal,et al.  Human Cortical Regions Involved in Extracting Depth from Motion , 1999, Neuron.

[16]  G. Orban,et al.  Selectivity for 3D shape that reveals distinct areas within macaque inferior temporal cortex. , 2000, Science.

[17]  G A Orban,et al.  Attention-dependent suppression of metabolic activity in the early stages of the macaque visual system. , 2000, Cerebral cortex.

[18]  A Berthoz,et al.  Visual perception of motion and 3-D structure from motion: an fMRI study. , 2000, Cerebral cortex.

[19]  N. Kanwisher,et al.  Cortical Regions Involved in Perceiving Object Shape , 2000, The Journal of Neuroscience.

[20]  G. Orban,et al.  Macaque Inferior Temporal Neurons Are Selective for Three-Dimensional Boundaries and Surfaces , 2001, The Journal of Neuroscience.

[21]  G. Orban,et al.  Visual Motion Processing Investigated Using Contrast Agent-Enhanced fMRI in Awake Behaving Monkeys , 2001, Neuron.

[22]  J T Todd,et al.  Ambiguity and the ‘Mental Eye’ in Pictorial Relief , 2001, Perception.

[23]  M. Sereno,et al.  Mapping of Contralateral Space in Retinotopic Coordinates by a Parietal Cortical Area in Humans , 2001, Science.

[24]  David J. Fleet,et al.  Human cortical activity correlates with stereoscopic depth perception. , 2001, Journal of neurophysiology.

[25]  David C. Van Essen,et al.  Application of Information Technology: An Integrated Software Suite for Surface-based Analyses of Cerebral Cortex , 2001, J. Am. Medical Informatics Assoc..

[26]  M. Taira,et al.  Cortical Areas Related to Attention to 3D Surface Structures Based on Shading: An fMRI Study , 2001, NeuroImage.

[27]  T. Hendler,et al.  Convergence of visual and tactile shape processing in the human lateral occipital complex. , 2002, Cerebral cortex.

[28]  Neurosciences,et al.  Organization of Visual Areas in Macaque and Human Cerebral Cortex , 2002 .

[29]  G. Orban,et al.  Extracting 3D from Motion: Differences in Human and Monkey Intraparietal Cortex , 2002, Science.

[30]  Thomas E. Nichols,et al.  Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate , 2002, NeuroImage.

[31]  S. Grossberg,et al.  A laminar cortical model of stereopsis and three-dimensional surface perception , 2003, Vision Research.

[32]  G. Orban,et al.  At Least at the Level of Inferior Temporal Cortex, the Stereo Correspondence Problem Is Solved , 2003, Neuron.

[33]  Doris Y. Tsao,et al.  Stereopsis Activates V3A and Caudal Intraparietal Areas in Macaques and Humans , 2003, Neuron.

[34]  W. Penny,et al.  ANOVAs and SPM , 2003 .

[35]  R. E Passingham,et al.  Activations related to “mirror” and “canonical” neurones in the human brain: an fMRI study , 2003, NeuroImage.

[36]  Guy A. Orban,et al.  Similarities and differences in motion processing between the human and macaque brain: evidence from fMRI , 2003, Neuropsychologia.

[37]  Bruno A Olshausen,et al.  Processing shape, motion and three-dimensional shape-from-motion in the human cortex. , 2003, Cerebral cortex.

[38]  Ravi S. Menon,et al.  Visually guided grasping produces fMRI activation in dorsal but not ventral stream brain areas , 2003, Experimental Brain Research.

[39]  G. Orban,et al.  A Higher Order Motion Region in Human Inferior Parietal Lobule Evidence from fMRI , 2003, Neuron.

[40]  Astrid M L Kappers,et al.  The Perception of Doubly Curved Surfaces From Anisotropic Textures , 2004, Psychological science.

[41]  G. Orban,et al.  Attention to 3-D Shape, 3-D Motion, and Texture in 3-D Structure from Motion Displays , 2004, Journal of Cognitive Neuroscience.

[42]  G. Rizzolatti,et al.  Functional organization of inferior area 6 in the macaque monkey , 2004, Experimental Brain Research.

[43]  David J Heeger,et al.  Stereoscopic processing of absolute and relative disparity in human visual cortex. , 2004, Journal of neurophysiology.

[44]  D. V. van Essen,et al.  The Processing of Visual Shape in the Cerebral Cortex of Human and Nonhuman Primates: A Functional Magnetic Resonance Imaging Study , 2004, The Journal of Neuroscience.

[45]  G. Orban,et al.  Comparative mapping of higher visual areas in monkeys and humans , 2004, Trends in Cognitive Sciences.

[46]  M. Sereno,et al.  From monkeys to humans: what do we now know about brain homologies? , 2005, Current Opinion in Neurobiology.

[47]  H. Bülthoff,et al.  3D shape perception from combined depth cues in human visual cortex , 2005, Nature Neuroscience.

[48]  Svetlana S. Georgieva,et al.  Using Functional Magnetic Resonance Imaging to Assess Adaptation and Size Invariance of Shape Processing by Humans and Monkeys , 2005, The Journal of Neuroscience.

[49]  Scott T. Grafton,et al.  Cortical topography of human anterior intraparietal cortex active during visually guided grasping. , 2005, Brain research. Cognitive brain research.

[50]  G. Orban,et al.  Observing Others: Multiple Action Representation in the Frontal Lobe , 2005, Science.

[51]  D. V. van Essen,et al.  A Population-Average, Landmark- and Surface-based (PALS) atlas of human cerebral cortex. , 2005, NeuroImage.

[52]  Stephen Grossberg,et al.  A laminar cortical model of stereopsis and 3D surface perception: closure and da Vinci stereopsis. , 2004, Spatial vision.

[53]  L. Fogassi,et al.  Functional properties of grasping-related neurons in the ventral premotor area F5 of the macaque monkey. , 2006, Journal of neurophysiology.

[54]  G. Luppino,et al.  Cortical connections of the inferior parietal cortical convexity of the macaque monkey. , 2006, Cerebral cortex.

[55]  G. Orban,et al.  Charting the Lower Superior Temporal Region, a New Motion-Sensitive Region in Monkey Superior Temporal Sulcus , 2006, The Journal of Neuroscience.

[56]  D. Heeger,et al.  Two Retinotopic Visual Areas in Human Lateral Occipital Cortex , 2006, The Journal of Neuroscience.

[57]  Steven W. Zucker,et al.  Differential Geometric Consistency Extends Stereo to Curved Surfaces , 2006, ECCV.

[58]  Guy A. Orban,et al.  Mapping the parietal cortex of human and non-human primates , 2006, Neuropsychologia.

[59]  Peter Janssen,et al.  Selectivity for three-dimensional shape in macaque posterior parietal cortex , 2006 .

[60]  M. Goodale,et al.  FMRI Reveals a Dissociation between Grasping and Perceiving the Size of Real 3D Objects , 2007, PloS one.

[61]  B. Wandell,et al.  Visual Field Maps in Human Cortex , 2007, Neuron.

[62]  Lotfi B Merabet,et al.  Visual Topography of Human Intraparietal Sulcus , 2007, The Journal of Neuroscience.

[63]  Kathleen A. Hansen,et al.  Topographic Organization in and near Human Visual Area V4 , 2007, The Journal of Neuroscience.

[64]  Peter Janssen,et al.  Coding for first- and second order disparity in macaque posterior parietal cortex , 2007 .

[65]  U. Castiello,et al.  Differential cortical activity for precision and whole‐hand visually guided grasping in humans , 2007, The European journal of neuroscience.

[66]  G. Orban,et al.  Somatotopy versus Actinotopy in human parietal and premotor cortex , 2007 .

[67]  Guy Orban,et al.  Visual 3D shape processing from disparity in the frontal lobe: fMRI evidence from awake monkeys , 2007 .

[68]  William M. Stern,et al.  Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex , 2007, Nature Neuroscience.

[69]  Peter Janssen,et al.  Anterior Regions of Monkey Parietal Cortex Process Visual 3D Shape , 2007, Neuron.

[70]  J. Culham,et al.  What does the brain do when you fake it? An FMRI study of pantomimed and real grasping. , 2007, Journal of neurophysiology.

[71]  Zoe Kourtzi,et al.  Neural correlates of disparity-defined shape discrimination in the human brain. , 2007, Journal of neurophysiology.

[72]  Guy A. Orban,et al.  The Extraction of 3D Shape from Texture and Shading in the Human Brain , 2008, Cerebral cortex.

[73]  S. Kastner,et al.  Two hierarchically organized neural systems for object information in human visual cortex , 2008, Nature Neuroscience.

[74]  Jascha D. Swisher,et al.  Behavioral / Systems / Cognitive Visual Topography of Human Intraparietal Sulcus , 2008 .

[75]  M. Sereno,et al.  Retinotopy and Attention in Human Occipital, Temporal, Parietal, and Frontal Cortex , 2008 .