Perceptual Integration for Qualitatively Different 3-D Cues in the Human Brain

The visual system's flexibility in estimating depth is remarkable: We readily perceive 3-D structure under diverse conditions from the seemingly random dots of a “magic eye” stereogram to the aesthetically beautiful, but obviously flat, canvasses of the Old Masters. Yet, 3-D perception is often enhanced when different cues specify the same depth. This perceptual process is understood as Bayesian inference that improves sensory estimates. Despite considerable behavioral support for this theory, insights into the cortical circuits involved are limited. Moreover, extant work tested quantitatively similar cues, reducing some of the challenges associated with integrating computationally and qualitatively different signals. Here we address this challenge by measuring fMRI responses to depth structures defined by shading, binocular disparity, and their combination. We quantified information about depth configurations (convex “bumps” vs. concave “dimples”) in different visual cortical areas using pattern classification analysis. We found that fMRI responses in dorsal visual area V3B/KO were more discriminable when disparity and shading concurrently signaled depth, in line with the predictions of cue integration. Importantly, by relating fMRI and psychophysical tests of integration, we observed a close association between depth judgments and activity in this area. Finally, using a cross-cue transfer test, we found that fMRI responses evoked by one cue afford classification of responses evoked by the other. This reveals a generalized depth representation in dorsal visual cortex that combines qualitatively different information in line with 3-D perception.

[1]  R. Tootell,et al.  Where is 'dorsal V4' in human visual cortex? Retinotopic, topographic and functional evidence. , 2001, Cerebral cortex.

[2]  Charless C. Fowlkes,et al.  Natural-Scene Statistics Predict How the Figure–Ground Cue of Convexity Affects Human Depth Perception , 2010, The Journal of Neuroscience.

[3]  Guy A. Orban,et al.  The Extraction of 3D Shape from Texture and Shading in the Human Brain , 2008, Cerebral cortex.

[4]  Peng Sun,et al.  What is second-order vision for? Discriminating illumination versus material changes. , 2010, Journal of vision.

[5]  Izumi Ohzawa,et al.  Joint-encoding of motion and depth by visual cortical neurons: neural basis of the Pulfrich effect , 2001, Nature Neuroscience.

[6]  Hiroshi Ban,et al.  The integration of motion and disparity cues to depth in dorsal visual cortex , 2012, Nature Neuroscience.

[7]  Nikos K. Logothetis,et al.  Three-Dimensional Shape Representation in Monkey Cortex , 2002, Neuron.

[8]  Rainer Goebel,et al.  Combining multivariate voxel selection and support vector machines for mapping and classification of fMRI spatial patterns , 2008, NeuroImage.

[9]  V. S. Ramachandran,et al.  Perception of shape from shading , 1988, Nature.

[10]  J. Koenderink,et al.  The shading cue in context , 2010, i-Perception.

[11]  Z Kourtzi,et al.  Representation of Perceived Object Shape by the Human Lateral Occipital Complex , 2001, Science.

[12]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[13]  James T Todd,et al.  Perceptual biases in the interpretation of 3D shape from shading , 2004, Vision Research.

[14]  Karl J. Friston,et al.  A direct demonstration of functional specialization in human visual cortex , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[15]  G. Sperling,et al.  Tradeoffs between stereopsis and proximity luminance covariance as determinants of perceived 3D structure , 1986, Vision Research.

[16]  Michael S Landy,et al.  Orientation selectivity of motion-boundary responses in human visual cortex. , 2010, Journal of neurophysiology.

[17]  Young Lim Lee,et al.  Stereo improves 3D shape discrimination even when rich monocular shape cues are available. , 2011, Journal of vision.

[18]  G. Orban,et al.  The kinetic occipital region in human visual cortex. , 1997, Cerebral cortex.

[19]  Marko Nardini,et al.  Interactions between "light-from-above" and convexity priors in visual development. , 2010, Journal of vision.

[20]  Zoe Kourtzi,et al.  Adaptive Estimation of Three-Dimensional Structure in the Human Brain , 2009, The Journal of Neuroscience.

[21]  H H Bülthoff,et al.  Integration of depth modules: stereo and shading. , 1988, Journal of the Optical Society of America. A, Optics and image science.

[22]  E. DeYoe,et al.  Mapping striate and extrastriate visual areas in human cerebral cortex. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[23]  Pete R. Jones,et al.  Development of Cue Integration in Human Navigation , 2008, Current Biology.

[24]  Jan J. Koenderink,et al.  The combined influence of binocular disparity and shading on pictorial shape , 2001, Perception & psychophysics.

[25]  Frederick A A Kingdom,et al.  Color brings relief to human vision , 2003, Nature Neuroscience.

[26]  M. Taira,et al.  Cortical Areas Related to Attention to 3D Surface Structures Based on Shading: An fMRI Study , 2001, NeuroImage.

[27]  Guy A. Orban,et al.  The Extraction of Depth Structure from Shading and Texture in the Macaque Brain , 2009, PloS one.

[28]  Berthold K. P. Horn Obtaining shape from shading information , 1989 .

[29]  J. Saunders,et al.  Do humans optimally integrate stereo and texture information for judgments of surface slant? , 2003, Vision Research.

[30]  Alex R. Wade,et al.  Extended Concepts of Occipital Retinotopy , 2005 .

[31]  S. Zeki,et al.  The processing of kinetic contours in the brain. , 2003, Cerebral cortex.

[32]  John M. Findlay,et al.  The area of spatial integration for initial horizontal disparity vergence , 1998, Vision Research.

[33]  G. Orban,et al.  Three-Dimensional Shape Coding in Inferior Temporal Cortex , 2000, Neuron.

[34]  K. Grill-Spector,et al.  The dynamics of object-selective activation correlate with recognition performance in humans , 2000, Nature Neuroscience.

[35]  Bosco S. Tjan,et al.  Efficient integration across spatial frequencies for letter identification in foveal and peripheral vision. , 2008, Journal of vision.

[36]  Z. Kourtzi,et al.  Multivoxel Pattern Selectivity for Perceptually Relevant Binocular Disparities in the Human Brain , 2008, The Journal of Neuroscience.

[37]  W Richards,et al.  Structure from stereo and motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[38]  Andrew Blake,et al.  Surface descriptions from stereo and shading , 1986, Image Vis. Comput..

[39]  Peter H. Schiller,et al.  The integration of disparity, shading and motion parallax cues for depth perception in humans and monkeys , 2011, Brain Research.

[40]  A. Yuille,et al.  Object perception as Bayesian inference. , 2004, Annual review of psychology.

[41]  Julie M. Harris,et al.  Optimal integration of shading and binocular disparity for depth perception. , 2012, Journal of vision.

[42]  Peter Janssen,et al.  A Distinct Representation of Three-Dimensional Shape in Macaque Anterior Intraparietal Area: Fast, Metric, and Coarse , 2009, The Journal of Neuroscience.

[43]  Sarah H. Creem-Regehr,et al.  Visual Perception from a Computer Graphics Perspective , 2011 .

[44]  M. Ernst Learning to integrate arbitrary signals from vision and touch. , 2007, Journal of vision.

[45]  David J. Kriegman,et al.  The Bas-Relief Ambiguity , 2004, International Journal of Computer Vision.

[46]  Fulvio Domini,et al.  Disparity and Shading Cues Cooperate for Surface Interpolation , 2006, Perception.

[47]  Mark A. Georgeson,et al.  Sun and sky: Does human vision assume a mixture of point and diffuse illumination when interpreting shape-from-shading? , 2011, Vision Research.

[48]  J W Belliveau,et al.  Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. , 1995, Science.

[49]  Roland W Fleming,et al.  Real-world illumination and the perception of surface reflectance properties. , 2003, Journal of vision.

[50]  E. Mingolla,et al.  Perception of solid shape from shading , 1989, Biological Cybernetics.

[51]  Alex R. Wade,et al.  The specificity of cortical region KO to depth structure , 2006, NeuroImage.

[52]  Pascal Mamassian,et al.  Prior knowledge of illumination for 3D perception in the human brain , 2010, Proceedings of the National Academy of Sciences.

[53]  A. Dale,et al.  The Retinotopy of Visual Spatial Attention , 1998, Neuron.

[54]  William Curran,et al.  The effect of Illuminant position on perceived curvature , 1996, Vision Research.

[55]  James M. Hillis,et al.  Combining Sensory Information: Mandatory Fusion Within, but Not Between, Senses , 2002, Science.

[56]  Gregory C DeAngelis,et al.  Coding of horizontal disparity and velocity by MT neurons in the alert macaque. , 2003, Journal of neurophysiology.

[57]  R. Jacobs,et al.  Experience-dependent visual cue integration based on consistencies between visual and haptic percepts , 2001, Vision Research.

[58]  M. Nardini,et al.  Fusion of visual cues is not mandatory in children , 2010, Proceedings of the National Academy of Sciences.

[59]  M. Landy,et al.  Measurement and modeling of depth cue combination: in defense of weak fusion , 1995, Vision Research.

[60]  Jeanine K. Stefanucci Illumination, Shading and Shadows , 2011 .

[61]  R. Andersen,et al.  Integration of motion and stereopsis in middle temporal cortical area of macaques , 1995, Nature.

[62]  Geoffrey M Boynton,et al.  The Representation of Behavioral Choice for Motion in Human Visual Cortex , 2007, The Journal of Neuroscience.

[63]  David C. Burr,et al.  Young Children Do Not Integrate Visual and Haptic Form Information , 2008, Current Biology.

[64]  P. Mamassian,et al.  Prior knowledge on the illumination position , 2001, Cognition.

[65]  D. Todd,et al.  The Sun , 1870, Nature.

[66]  A. Pentland Finding the illuminant direction , 1982 .