Generalized Representation of Stereoscopic Surface Shape and Orientation in the Human Visual Cortex

The brain’s ability to extract three-dimensional (3D) shape and orientation information from viewed objects is vital in daily life. Stereoscopic 3D surface perception relies on binocular disparity. Neurons selective to binocular disparity are widely distributed among visual areas, but the manner in these areas are involved in stereoscopic 3D surface representation is unclear. To address this, participants were instructed to observe random dot stereograms (RDS) depicting convex and concave curved surfaces and the blood oxygenation level-dependent (BOLD) signal of visual cortices was recorded. Two surface types were: (i) horizontally positioned surfaces defined by shear disparity; and (ii) vertically positioned surfaces defined by compression disparity. The surfaces were presented at different depth positions per trial. Functional magnetic resonance imaging (fMRI) data were classified from early visual areas to higher visual areas. We determined whether cortical areas were selective to shape and orientation by assessing same-type stimuli classification accuracies based on multi-voxel activity patterns per area. To identify whether some areas were related to a more generalized sign of curvature or orientation representation, transfer classification was used by training classifiers on one dataset type and testing classifiers on another type. Same-type stimuli classification results showed that most selected visual areas were selective to shape and all were selective to the orientation of disparity-defined 3D surfaces. Transfer classification results showed that in the dorsal visual area V3A, classification accuracies for the discriminate sign of surface curvature were higher than the baseline of statistical significance for all types of classifications, demonstrating that V3A is related to generalized shape representation. Classification accuracies for discriminating horizontal–vertical surfaces in higher dorsal areas V3A and V7 and ventral area lateral occipital complex (LOC) as well as in some areas of intraparietal sulcus (IPS) were higher than the baseline of statistical significance, indicating their relation to the generalized representation of 3D surface orientation.

[1]  Jerry D. Nguyenkim,et al.  Disparity-Based Coding of Three-Dimensional Surface Orientation by Macaque Middle Temporal Neurons , 2003, The Journal of Neuroscience.

[2]  Peter Janssen,et al.  Posterior Parietal Cortex Drives Inferotemporal Activations During Three-Dimensional Object Vision , 2016, PLoS biology.

[3]  A. Parker.  Binocular depth perception and the cerebral cortex , 2007, Nature Reviews Neuroscience.

[4]  Zoe Kourtzi,et al.  Neural correlates of disparity-defined shape discrimination in the human brain. , 2007, Journal of neurophysiology.

[5]  H. Sakata,et al.  Parietal neurons represent surface orientation from the gradient of binocular disparity. , 2000, Journal of neurophysiology.

[6]  C. Büchel,et al.  Surface orientation discrimination activates caudal and anterior intraparietal sulcus in humans: an event-related fMRI study. , 2001, Journal of neurophysiology.

[7]  Peter Neri,et al.  A stereoscopic look at visual cortex. , 2005, Journal of neurophysiology.

[8]  Andrew E. Welchman,et al.  fMRI Activity in Posterior Parietal Cortex Relates to the Perceptual Use of Binocular Disparity for Both Signal-In-Noise and Feature Difference Tasks , 2015, PloS one.

[9]  G. Orban,et al.  Extracting 3D from Motion: Differences in Human and Monkey Intraparietal Cortex , 2002, Science.

[10]  G. Orban,et al.  Three-Dimensional Shape Coding in Inferior Temporal Cortex , 2000, Neuron.

[11]  Tomoyasu Horikawa,et al.  Generic decoding of seen and imagined objects using hierarchical visual features , 2015, Nature Communications.

[12]  A Berthoz,et al.  Visual perception of motion and 3-D structure from motion: an fMRI study. , 2000, Cerebral cortex.

[13]  Ari Rosenberg,et al.  The Visual Representation of 3D Object Orientation in Parietal Cortex , 2013, The Journal of Neuroscience.

[14]  Hiroshi Ban,et al.  fMRI Analysis-by-Synthesis Reveals a Dorsal Hierarchy That Extracts Surface Slant , 2015, The Journal of Neuroscience.

[15]  S. Zeki,et al.  The processing of kinetic contours in the brain. , 2003, Cerebral cortex.

[16]  Bevil R. Conway,et al.  Binocular stereoscopy in visual areas V-2, V-3, and V-3A of the macaque monkey. , 2015, Cerebral cortex.

[17]  A. Dale,et al.  The Retinotopy of Visual Spatial Attention , 1998, Neuron.

[18]  N. Logothetis,et al.  Neurophysiological investigation of the basis of the fMRI signal , 2001, Nature.

[19]  Yannis E. Ioannidis,et al.  Visual Representation , 2009, Encyclopedia of Database Systems.

[20]  Nikos K. Logothetis,et al.  Three-Dimensional Shape Representation in Monkey Cortex , 2002, Neuron.

[21]  R Vogels,et al.  Macaque inferior temporal neurons are selective for disparity-defined three-dimensional shapes. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[22]  K Tsutsui,et al.  Neural coding of 3D features of objects for hand action in the parietal cortex of the monkey. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[23]  M. Goodale,et al.  Separate visual pathways for perception and action , 1992, Trends in Neurosciences.

[24]  G. Orban,et al.  Macaque Inferior Temporal Neurons Are Selective for Three-Dimensional Boundaries and Surfaces , 2001, The Journal of Neuroscience.

[25]  G C DeAngelis,et al.  The physiology of stereopsis. , 2001, Annual review of neuroscience.

[26]  Andrea J. van Doorn,et al.  Surface shape and curvature scales , 1992, Image Vis. Comput..

[27]  J W Belliveau,et al.  Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. , 1995, Science.

[28]  Hiroshi Ban,et al.  Integration of texture and disparity cues to surface slant in dorsal visual cortex. , 2013, Journal of neurophysiology.

[29]  Tomoka Naganuma,et al.  Information processing of geometrical features of a surface based on binocular disparity cues: an fMRI study , 2005, Neuroscience Research.

[30]  N. Logothetis,et al.  Role of the color-opponent and broad-band channels in vision , 1990, Visual Neuroscience.

[31]  D Marr,et al.  Cooperative computation of stereo disparity. , 1976, Science.

[32]  Scott T. Grafton,et al.  Automated image registration: I. General methods and intrasubject, intramodality validation. , 1998, Journal of computer assisted tomography.

[33]  R. Hogg,et al.  On adaptive estimation , 1984 .

[34]  Guy A. Orban,et al.  The Extraction of 3D Shape from Texture and Shading in the Human Brain , 2008, Cerebral cortex.

[35]  B. Wandell,et al.  The vertical occipital fasciculus: A century of controversy resolved by in vivo measurements , 2014, Proceedings of the National Academy of Sciences.

[36]  S. R. Lehky,et al.  Comparison of shape encoding in primate dorsal and ventral visual pathways. , 2007, Journal of neurophysiology.

[37]  G. DeAngelis,et al.  Coding of Stereoscopic Depth Information in Visual Areas V3 and V3A , 2011, The Journal of Neuroscience.

[38]  Alex R. Wade,et al.  Extended Concepts of Occipital Retinotopy , 2005 .

[39]  Svetlana S. Georgieva,et al.  The Processing of Three-Dimensional Shape from Disparity in the Human Brain , 2009, The Journal of Neuroscience.

[40]  Karl J. Friston,et al.  A direct demonstration of functional specialization in human visual cortex , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[41]  G. Orban,et al.  The kinetic occipital region in human visual cortex. , 1997, Cerebral cortex.

[42]  Peter J. Passmore,et al.  Independent encoding of surface orientation an surface curvature , 1994, Vision Research.

[43]  Zoe Kourtzi,et al.  Adaptive Estimation of Three-Dimensional Structure in the Human Brain , 2009, The Journal of Neuroscience.

[44]  A. Parker,et al.  A specialization for relative disparity in V2 , 2002, Nature Neuroscience.

[45]  Gregory C. DeAngelis,et al.  Behavioral / Systems / Cognitive Coding of Stereoscopic Depth Information in Visual Areas V 3 and V 3 A , 2011 .

[46]  H. Sakata,et al.  Integration of perspective and disparity cues in surface-orientation-selective neurons of area CIP. , 2001, Journal of neurophysiology.

[47]  Palmer Taylor,et al.  Intersubunit contacts governing assembly of the mammalian nicotinic acetylcholine receptor , 1995, Neuron.

[48]  H. Sakata,et al.  Selectivity of the parietal visual neurones in 3D orientation of surface of stereoscopic stimuli. , 1996, Neuroreport.

[49]  B. G. Cumming,et al.  Responses of primary visual cortical neurons to binocular disparity without depth perception , 1997, Nature.

[50]  Peter Janssen,et al.  Single-cell responses to three-dimensional structure in a functionally defined patch in macaque area TEO. , 2018, Journal of neurophysiology.

[51]  G. DeAngelis,et al.  Linking Neural Representation to Function in Stereoscopic Depth Perception: Roles of the Middle Temporal Area in Coarse versus Fine Disparity Discrimination , 2006, The Journal of Neuroscience.

[52]  Chantal Delon-Martin,et al.  fMRI Retinotopic Mapping—Step by Step , 2002, NeuroImage.

[53]  E. DeYoe,et al.  Mapping striate and extrastriate visual areas in human cerebral cortex. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[54]  Guy Marchal,et al.  Human Cortical Regions Involved in Extracting Depth from Motion , 1999, Neuron.

[55]  Bruce Fischl,et al.  FreeSurfer , 2012, NeuroImage.

[56]  M. Taira,et al.  Cortical Areas Related to Attention to 3D Surface Structures Based on Shading: An fMRI Study , 2001, NeuroImage.

[57]  Gregory C DeAngelis,et al.  Disparity Channels in Early Vision , 2007, The Journal of Neuroscience.

[58]  M. Behrmann,et al.  Three‐Dimensional Representations of Objects in Dorsal Cortex are Dissociable from Those in Ventral Cortex , 2017, Cerebral cortex.

[59]  Peter Janssen,et al.  Extracting 3D structure from disparity , 2006, Trends in Neurosciences.

[60]  Leslie G. Ungerleider,et al.  Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys , 1982, Behavioural Brain Research.

[61]  Stephen Grossberg,et al.  On the road to invariant object recognition: How cortical area V2 transforms absolute to relative disparity during 3D vision , 2011, Neural Networks.

[62]  David J Heeger,et al.  Stereoscopic processing of absolute and relative disparity in human visual cortex. , 2004, Journal of neurophysiology.

[63]  Denis Schluppeck,et al.  7 Tesla fMRI Reveals Systematic Functional Organization for Binocular Disparity in Dorsal Visual Cortex , 2015, The Journal of Neuroscience.

[64]  Z Kourtzi,et al.  Representation of Perceived Object Shape by the Human Lateral Occipital Complex , 2001, Science.

[65]  Jonathan Winawer,et al.  A Major Human White Matter Pathway Between Dorsal and Ventral Visual Cortex. , 2016, Cerebral cortex.

[66]  Tomoka Naganuma,et al.  Neural Correlates for Perception of 3D Surface Orientation from Texture Gradient , 2002, Science.

[67]  N. Kanwisher,et al.  Cortical Regions Involved in Perceiving Object Shape , 2000, The Journal of Neuroscience.

[68]  Guy A. Orban,et al.  Mapping the parietal cortex of human and non-human primates , 2006, Neuropsychologia.

[69]  S. Zeki.  Functional specialisation in the visual cortex of the rhesus monkey , 1978, Nature.

[70]  S. Kastner,et al.  Two hierarchically organized neural systems for object information in human visual cortex , 2008, Nature Neuroscience.

[71]  Z. Kourtzi,et al.  Multivoxel Pattern Selectivity for Perceptually Relevant Binocular Disparities in the Human Brain , 2008, The Journal of Neuroscience.

[72]  Markus Graf,et al.  When Action Observation Facilitates Visual Perception: Activation in Visuo-Motor Areas Contributes to Object Recognition. , 2015, Cerebral cortex.

[73]  Christian Leibold,et al.  Decoding cognitive concepts from neuroimaging data using multivariate pattern analysis , 2017, NeuroImage.

[74]  Hiroshi Ban,et al.  Perceptual Integration for Qualitatively Different 3-D Cues in the Human Brain , 2013, Journal of Cognitive Neuroscience.