Integration of texture and disparity cues to surface slant in dorsal visual cortex.

Reliable estimation of three-dimensional (3D) surface orientation is critical for recognizing and interacting with complex 3D objects in our environment. Human observers maximize the reliability of their estimates of surface slant by integrating multiple depth cues. Texture and binocular disparity are two such cues, but they are qualitatively very different. Existing evidence suggests that representations of surface tilt from each of these cues coincide at the single-neuron level in higher cortical areas. However, the cortical circuits responsible for 1) integration of such qualitatively distinct cues and 2) encoding the slant component of surface orientation have not been assessed. We tested for cortical responses related to slanted plane stimuli that were defined independently by texture, disparity, and combinations of these two cues. We analyzed the discriminability of functional MRI responses to two slant angles using multivariate pattern classification. Responses in visual area V3B/KO to stimuli containing congruent cues were more discriminable than those elicited by single cues, in line with predictions based on the fusion of slant estimates from component cues. This improvement was specific to congruent combinations of cues: incongruent cues yielded lower decoding accuracies, which suggests the robust use of individual cues in cases of large cue conflicts. These data suggest that area V3B/KO is intricately involved in the integration of qualitatively dissimilar depth cues.

[1]  A. Dale,et al.  The Retinotopy of Visual Spatial Attention , 1998, Neuron.

[2]  A. Watson,et al.  Quest: A Bayesian adaptive psychometric method , 1983, Perception & psychophysics.

[3]  Guy Marchal,et al.  Human Cortical Regions Involved in Extracting Depth from Motion , 1999, Neuron.

[4]  Marko Nardini,et al.  Fusion of disparity and texture cues to slant is not mandatory in children , 2010 .

[5]  David C. Knill,et al.  Surface orientation from texture: ideal observers, generic observers and the information content of texture cues , 1998, Vision Research.

[6]  M. Braunstein Motion and texture as sources of slant information. , 1968, Journal of experimental psychology.

[7]  James M. Hillis,et al.  Combining Sensory Information: Mandatory Fusion Within, but Not Between, Senses , 2002, Science.

[8]  S. Zeki,et al.  The processing of kinetic contours in the brain. , 2003, Cerebral cortex.

[9]  J. Hennig,et al.  Functional magnetic resonance imaging evidence for binocular interactions in human visual cortex , 2002, Experimental Brain Research.

[10]  G. Orban,et al.  The kinetic occipital region in human visual cortex. , 1997, Cerebral cortex.

[11]  Andrew E. Welchman Decoding the Cortical Representation of Depth , 2011 .

[12]  Rainer Goebel,et al.  Combining multivariate voxel selection and support vector machines for mapping and classification of fMRI spatial patterns , 2008, NeuroImage.

[13]  Aldo Genovesio,et al.  Integration of retinal disparity and fixation-distance related signals toward an egocentric coding of distance in the posterior parietal cortex of primates. , 2004, Journal of neurophysiology.

[14]  G C DeAngelis,et al.  The physiology of stereopsis. , 2001, Annual review of neuroscience.

[15]  James M. Hillis,et al.  Slant from texture and disparity cues: optimal cue combination. , 2004, Journal of vision.

[16]  M. Arbib,et al.  Grasping objects: the cortical mechanisms of visuomotor transformation , 1995, Trends in Neurosciences.

[17]  Rufin Vogels,et al.  Convergence of Depth from Texture and Depth from Disparity in Macaque Inferior Temporal Cortex , 2004, The Journal of Neuroscience.

[18]  Nikolaus Kriegeskorte,et al.  How does an fMRI voxel sample the neuronal activity pattern: Compact-kernel or complex spatiotemporal filter? , 2010, NeuroImage.

[19]  J. Wagemans,et al.  Some observations on the effects of slant and texture type on slant-from-texture , 2004, Vision Research.

[20]  Doris Y. Tsao,et al.  Stereopsis Activates V3A and Caudal Intraparietal Areas in Macaques and Humans , 2003, Neuron.

[21]  R. van Ee,et al.  Activation in Visual Cortex Correlates with the Awareness of Stereoscopic Depth , 2005 .

[22]  B. Rogers,et al.  Similarities between motion parallax and stereopsis in human depth perception , 1982, Vision Research.

[23]  J. Gibson The Ecological Approach to the Visual Perception of Pictures , 1978 .

[24]  Volkmar Glauche,et al.  Localization of human intraparietal areas AIP, CIP, and LIP using surface orientation and saccadic eye movement tasks , 2008, Human brain mapping.

[25]  Geoffrey M Boynton,et al.  The Representation of Behavioral Choice for Motion in Human Visual Cortex , 2007, The Journal of Neuroscience.

[26]  Kent A. Stevens,et al.  Slant-tilt: The visual encoding of surface orientation , 1983, Biological Cybernetics.

[27]  Geraint Rees,et al.  Knowing with Which Eye We See: Utrocular Discrimination and Eye-Specific Signals in Human Visual Cortex , 2010, PloS one.

[28]  Simon B. Eickhoff,et al.  Comparison of functional and cytoarchitectonic maps of human visual areas V1, V2, V3d, V3v, and V4(v) , 2010, NeuroImage.

[29]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Felix Wichmann,et al.  The psychometric function: II. Bootstrap-based confidence intervals and sampling , 2001, Perception & psychophysics.

[31]  David C Knill,et al.  Mixture models and the probabilistic structure of depth cues , 2003, Vision Research.

[32]  K. Grill-Spector,et al.  The dynamics of object-selective activation correlate with recognition performance in humans , 2000, Nature Neuroscience.

[33]  M. Taira,et al.  Cortical Areas Related to Attention to 3D Surface Structures Based on Shading: An fMRI Study , 2001, NeuroImage.

[34]  F A Wichmann,et al.  Ning for Helpful Comments and Suggestions. This Paper Benefited Con- Siderably from Conscientious Peer Review, and We Thank Our Reviewers the Psychometric Function: I. Fitting, Sampling, and Goodness of Fit , 2001 .

[35]  R. Andersen,et al.  Response of MSTd neurons to simulated 3D orientation of rotating planes. , 2002, Journal of neurophysiology.

[36]  R. Tootell,et al.  Where is 'dorsal V4' in human visual cortex? Retinotopic, topographic and functional evidence. , 2001, Cerebral cortex.

[37]  Jerry D. Nguyenkim,et al.  Disparity-Based Coding of Three-Dimensional Surface Orientation by Macaque Middle Temporal Neurons , 2003, The Journal of Neuroscience.

[38]  Guy A. Orban,et al.  The Extraction of 3D Shape from Texture and Shading in the Human Brain , 2008, Cerebral cortex.

[39]  H. Sakata,et al.  Parietal neurons represent surface orientation from the gradient of binocular disparity. , 2000, Journal of neurophysiology.

[40]  E. DeYoe,et al.  Mapping striate and extrastriate visual areas in human cerebral cortex. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[41]  J. Gibson The perception of visual surfaces. , 1950, The American journal of psychology.

[42]  Lotfi B Merabet,et al.  Visual Topography of Human Intraparietal Sulcus , 2007, The Journal of Neuroscience.

[43]  J. Koenderink,et al.  Surface perception in pictures , 1992, Perception & psychophysics.

[44]  Zoe Kourtzi,et al.  Neural correlates of disparity-defined shape discrimination in the human brain. , 2007, Journal of neurophysiology.

[45]  E. Goldstein,et al.  Spatial layout, orientation relative to the observer, and perceived projection in pictures viewed at an angle. , 1987, Journal of experimental psychology. Human perception and performance.

[46]  Tomoka Naganuma,et al.  Neural Correlates for Perception of 3D Surface Orientation from Texture Gradient , 2002, Science.

[47]  J. Saunders,et al.  Do humans optimally integrate stereo and texture information for judgments of surface slant? , 2003, Vision Research.

[48]  Alex R. Wade,et al.  The specificity of cortical region KO to depth structure , 2006, NeuroImage.

[49]  Z Kourtzi,et al.  Representation of Perceived Object Shape by the Human Lateral Occipital Complex , 2001, Science.

[50]  Alex R. Wade,et al.  Extended Concepts of Occipital Retinotopy , 2005 .

[51]  J W Belliveau,et al.  Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. , 1995, Science.

[52]  Peter Janssen,et al.  A Distinct Representation of Three-Dimensional Shape in Macaque Anterior Intraparietal Area: Fast, Metric, and Coarse , 2009, The Journal of Neuroscience.

[53]  Frédéric Gosselin,et al.  Spatio-temporal use of information in face recognition , 2010 .

[54]  Leslie G. Ungerleider,et al.  Organization of visual inputs to the inferior temporal and posterior parietal cortex in macaques , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[55]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[56]  Goldstein Eb Spatial layout, orientation relative to the observer, and perceived projection in pictures viewed at an angle. , 1987 .

[57]  Zoe Kourtzi,et al.  Adaptive Estimation of Three-Dimensional Structure in the Human Brain , 2009, The Journal of Neuroscience.

[58]  G. Orban,et al.  Selectivity for 3D shape that reveals distinct areas within macaque inferior temporal cortex. , 2000, Science.

[59]  Barbara Gillam,et al.  Perspective, Orientation Disparity, and Anisotropy in Stereoscopic Slant Perception , 1992, Perception.

[60]  Richard A Andersen,et al.  Parietal reach region encodes reach depth using retinal disparity and vergence angle signals. , 2009, Journal of neurophysiology.

[61]  Hideko F. Norman,et al.  Visual discrimination of local surface structure: Slant, tilt, and curvedness , 2006, Vision Research.

[62]  M. Landy,et al.  Measurement and modeling of depth cue combination: in defense of weak fusion , 1995, Vision Research.

[63]  G. Rees,et al.  Predicting the orientation of invisible stimuli from activity in human primary visual cortex , 2005, Nature Neuroscience.

[64]  B. Julesz,et al.  A disparity gradient limit for binocular fusion. , 1980, Science.

[65]  Alex R. Wade,et al.  Visual areas and spatial summation in human visual cortex , 2001, Vision Research.

[66]  B. Cumming,et al.  Decision-related activity in sensory neurons reflects more than a neuron’s causal effect , 2009, Nature.

[67]  James A. Crowell,et al.  Horizontal and vertical disparity, eye position, and stereoscopic slant perception , 1999, Vision Research.

[68]  G. Orban The extraction of 3D shape in the visual system of human and nonhuman primates. , 2011, Annual review of neuroscience.

[69]  Peter Janssen,et al.  Synchronization between the end stages of the dorsal and the ventral visual stream. , 2011, Journal of neurophysiology.

[70]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[71]  G. Orban,et al.  Selectivity of Macaque MT/V5 Neurons for Surface Orientation in Depth Specified by Motion , 1997, The European journal of neuroscience.

[72]  Z. Kourtzi,et al.  Multivoxel Pattern Selectivity for Perceptually Relevant Binocular Disparities in the Human Brain , 2008, The Journal of Neuroscience.

[73]  H. Bülthoff,et al.  3D shape perception from combined depth cues in human visual cortex , 2005, Nature Neuroscience.

[74]  John M. Findlay,et al.  The area of spatial integration for initial horizontal disparity vergence , 1998, Vision Research.

[75]  K Tsutsui,et al.  Neural coding of 3D features of objects for hand action in the parietal cortex of the monkey. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[76]  Takahisa M. Sanada,et al.  Representation of 3-D surface orientation by velocity and disparity gradient cues in area MT. , 2012, Journal of Neurophysiology.

[77]  Lisa R. Betts,et al.  Distributed Neural Plasticity for Shape Learning in the Human Visual Cortex , 2005, PLoS biology.

[78]  C. Büchel,et al.  Surface orientation discrimination activates caudal and anterior intraparietal sulcus in humans: an event-related fMRI study. , 2001, Journal of neurophysiology.

[79]  Nikos K. Logothetis,et al.  Three-Dimensional Shape Representation in Monkey Cortex , 2002, Neuron.

[80]  Jitendra Malik,et al.  Surface orientation from texture: Isotropy or homogeneity (or both)? , 1997, Vision Research.

[81]  Svetlana S. Georgieva,et al.  The Processing of Three-Dimensional Shape from Disparity in the Human Brain , 2009, The Journal of Neuroscience.

[82]  F. A. Miles Binocular Vision and Stereopsis by Ian P. Howard and Brian J. Rogers, Oxford University Press, 1995. £90.00 (736 pages) ISBN 0 19 508476 4. , 1996, Trends in Neurosciences.

[83]  Yale E. Cohen,et al.  A common reference frame for movement plans in the posterior parietal cortex , 2002, Nature Reviews Neuroscience.

[84]  Karl J. Friston,et al.  A direct demonstration of functional specialization in human visual cortex , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[85]  David Marr,et al.  VISION A Computational Investigation into the Human Representation and Processing of Visual Information , 2009 .

[86]  Peter Janssen,et al.  Anterior Regions of Monkey Parietal Cortex Process Visual 3D Shape , 2007, Neuron.

[87]  Raymond van Ee,et al.  Temporal aspects of stereoscopic slant estimation: an evaluation and extension of Howard and Kaneko's theory , 1998, Vision Research.

[88]  Hiroshi Ban,et al.  The integration of motion and disparity cues to depth in dorsal visual cortex , 2012, Nature Neuroscience.

[89]  Peter Janssen,et al.  Selectivity for three-dimensional contours and surfaces in the anterior intraparietal area. , 2012, Journal of neurophysiology.

[90]  H H Bülthoff,et al.  Integration of depth modules: stereo and shading. , 1988, Journal of the Optical Society of America. A, Optics and image science.

[91]  James J. Clark,et al.  Data Fusion for Sensory Information Processing Systems , 1990 .

[92]  Ahna R Girshick,et al.  Probabilistic combination of slant information: weighted averaging and robustness as optimal percepts. , 2009, Journal of vision.

[93]  Izumi Ohzawa,et al.  Joint-encoding of motion and depth by visual cortical neurons: neural basis of the Pulfrich effect , 2001, Nature Neuroscience.

[94]  Ronald R. Peeters,et al.  Parietal regions processing visual 3D shape extracted from disparity , 2009, NeuroImage.

[95]  A. Parker Binocular depth perception and the cerebral cortex , 2007, Nature Reviews Neuroscience.

[96]  R Vogels,et al.  Macaque inferior temporal neurons are selective for disparity-defined three-dimensional shapes. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[97]  M. Torrens Co-Planar Stereotaxic Atlas of the Human Brain—3-Dimensional Proportional System: An Approach to Cerebral Imaging, J. Talairach, P. Tournoux. Georg Thieme Verlag, New York (1988), 122 pp., 130 figs. DM 268 , 1990 .

[98]  Raymond van Ee,et al.  Bi-stability in perceived slant when binocular disparity and monocular perspective specify different slants. , 2002, Journal of vision.

[99]  Guy A. Orban,et al.  Mapping the parietal cortex of human and non-human primates , 2006, Neuropsychologia.

[100]  H. Sakata,et al.  Integration of perspective and disparity cues in surface-orientation-selective neurons of area CIP. , 2001, Journal of neurophysiology.

[101]  H. Sakata,et al.  Neural representation of three-dimensional features of manipulation objects with stereopsis , 1999, Experimental Brain Research.

[102]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.