The integration of motion and disparity cues to depth in dorsal visual cortex

Humans exploit a range of visual depth cues to estimate three-dimensional structure. For example, the slant of a nearby tabletop can be judged by combining information from binocular disparity, texture and perspective. Behavioral tests show humans combine cues near-optimally, a feat that could depend on discriminating the outputs from cue-specific mechanisms or on fusing signals into a common representation. Although fusion is computationally attractive, it poses a substantial challenge, requiring the integration of quantitatively different signals. We used functional magnetic resonance imaging (fMRI) to provide evidence that dorsal visual area V3B/KO meets this challenge. Specifically, we found that fMRI responses are more discriminable when two cues (binocular disparity and relative motion) concurrently signal depth, and that information provided by one cue is diagnostic of depth indicated by the other. This suggests a cortical node important when perceiving depth, and highlights computations based on fusion in the dorsal stream.

[1]  Zoe Kourtzi,et al.  Neural correlates of disparity-defined shape discrimination in the human brain. , 2007, Journal of neurophysiology.

[2]  Essa Yacoub,et al.  High-field fMRI unveils orientation columns in humans , 2008, Proceedings of the National Academy of Sciences.

[3]  Peter Janssen,et al.  Extracting 3D structure from disparity , 2006, Trends in Neurosciences.

[4]  B. Rogers,et al.  The Interaction of Binocular Disparity and Motion Parallax in the Computation of Depth , 1996, Vision Research.

[5]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[6]  B. Stein,et al.  Interactions among converging sensory inputs in the superior colliculus. , 1983, Science.

[7]  Guy Marchal,et al.  Human Cortical Regions Involved in Extracting Depth from Motion , 1999, Neuron.

[8]  Gregory C. DeAngelis,et al.  A neural representation of depth from motion parallax in macaque visual cortex , 2008, Nature.

[9]  Fulvio Domini,et al.  Stereo and motion information are not independently processed by the visual system , 2006, Vision Research.

[10]  Zoe Kourtzi,et al.  Uncertainty and invariance in the human visual cortex. , 2006, Journal of neurophysiology.

[11]  John M. Findlay,et al.  The area of spatial integration for initial horizontal disparity vergence , 1998, Vision Research.

[12]  J. Saunders,et al.  Do humans optimally integrate stereo and texture information for judgments of surface slant? , 2003, Vision Research.

[13]  Alex R. Wade,et al.  The specificity of cortical region KO to depth structure , 2006, NeuroImage.

[14]  Tomoka Naganuma,et al.  Neural Correlates for Perception of 3D Surface Orientation from Texture Gradient , 2002, Science.

[15]  Rainer Goebel,et al.  Combining multivariate voxel selection and support vector machines for mapping and classification of fMRI spatial patterns , 2008, NeuroImage.

[16]  Scott O. Murray,et al.  Processing Shape, Motion and Three-dimensional Shape-from-motion in the Human Cortex , 2003 .

[17]  Rufin Vogels,et al.  Convergence of Depth from Texture and Depth from Disparity in Macaque Inferior Temporal Cortex , 2004, The Journal of Neuroscience.

[18]  Essa Yacoub,et al.  Mechanisms underlying decoding at 7 T: Ocular dominance columns, broad structures, and macroscopic blood vessels in V1 convey information on the stimulated eye , 2010, NeuroImage.

[19]  A Berthoz,et al.  Visual perception of motion and 3-D structure from motion: an fMRI study. , 2000, Cerebral cortex.

[20]  J. Duhamel,et al.  Multisensory Integration in the Ventral Intraparietal Area of the Macaque Monkey , 2007, The Journal of Neuroscience.

[21]  David J. Fleet,et al.  Human cortical activity correlates with stereoscopic depth perception. , 2001, Journal of neurophysiology.

[22]  Nikolaus Kriegeskorte,et al.  How does an fMRI voxel sample the neuronal activity pattern: Compact-kernel or complex spatiotemporal filter? , 2010, NeuroImage.

[23]  H H Bülthoff,et al.  Integration of depth modules: stereo and shading. , 1988, Journal of the Optical Society of America. A, Optics and image science.

[24]  M Nawrot,et al.  Neural integration of information specifying structure from stereopsis and motion. , 1989, Science.

[25]  G. Orban,et al.  The kinetic occipital region in human visual cortex. , 1997, Cerebral cortex.

[26]  Zoe Kourtzi,et al.  Adaptive Estimation of Three-Dimensional Structure in the Human Brain , 2009, The Journal of Neuroscience.

[27]  D. Heeger,et al.  Neuronal basis of contrast discrimination , 1999, Vision Research.

[28]  W. Newsome,et al.  A selective impairment of motion perception following lesions of the middle temporal visual area (MT) , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[29]  Kun Luo,et al.  Visualization of vortex shedding and particle dispersion in two-phase plate wake , 2005, J. Vis..

[30]  Bruno A Olshausen,et al.  Processing shape, motion and three-dimensional shape-from-motion in the human cortex. , 2003, Cerebral cortex.

[31]  G. DeAngelis,et al.  Organization of Disparity-Selective Neurons in Macaque Area MT , 1999, The Journal of Neuroscience.

[32]  A. Parker Binocular depth perception and the cerebral cortex , 2007, Nature Reviews Neuroscience.

[33]  M. Landy,et al.  Measurement and modeling of depth cue combination: in defense of weak fusion , 1995, Vision Research.

[34]  G. Orban,et al.  Attention to 3-D Shape, 3-D Motion, and Texture in 3-D Structure from Motion Displays , 2004, Journal of Cognitive Neuroscience.

[35]  G. DeAngelis,et al.  Neural correlates of multisensory cue integration in macaque MSTd , 2008, Nature Neuroscience.

[36]  Hans P. Op de Beeck,et al.  Against hyperacuity in brain reading: Spatial smoothing does not hurt multivariate fMRI analyses? , 2010, NeuroImage.

[37]  G. Sperling,et al.  Tradeoffs between stereopsis and proximity luminance covariance as determinants of perceived 3D structure , 1986, Vision Research.

[38]  Konrad Paul Kording,et al.  Causal Inference in Multisensory Perception , 2007, PloS one.

[39]  D. S. M A X W E L L,et al.  B Y H E a V Y I O N I Z I N G P a R T I C L E S , 2022 .

[40]  James J. Clark,et al.  Data Fusion for Sensory Information Processing Systems , 1990 .

[41]  James M. Hillis,et al.  Combining Sensory Information: Mandatory Fusion Within, but Not Between, Senses , 2002, Science.

[42]  Bosco S. Tjan,et al.  Efficient integration across spatial frequencies for letter identification in foveal and peripheral vision. , 2008, Journal of vision.

[43]  Eero P. Simoncelli,et al.  A model of neuronal responses in visual area MT , 1998, Vision Research.

[44]  Geoffrey M Boynton,et al.  The Representation of Behavioral Choice for Motion in Human Visual Cortex , 2007, The Journal of Neuroscience.

[45]  G. DeAngelis,et al.  Multisensory Integration in Macaque Visual Cortex Depends on Cue Reliability , 2008, Neuron.

[46]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[47]  Z. Kourtzi,et al.  Multivoxel Pattern Selectivity for Perceptually Relevant Binocular Disparities in the Human Brain , 2008, The Journal of Neuroscience.

[48]  Guy A. Orban,et al.  Similarities and differences in motion processing between the human and macaque brain: evidence from fMRI , 2003, Neuropsychologia.

[49]  H. Bülthoff,et al.  3D shape perception from combined depth cues in human visual cortex , 2005, Nature Neuroscience.

[50]  L. Poom,et al.  Perceptual depth synthesis in the visual system as revealed by selective adaptation. , 1999, Journal of experimental psychology. Human perception and performance.

[51]  Jeremy Freeman,et al.  Orientation Decoding Depends on Maps, Not Columns , 2011, The Journal of Neuroscience.

[52]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[53]  T. Stanford,et al.  Evaluating the Operations Underlying Multisensory Integration in the Cat Superior Colliculus , 2005, The Journal of Neuroscience.

[54]  B Rogers,et al.  Motion Parallax as an Independent Cue for Depth Perception , 1979, Perception.

[55]  Peter Janssen,et al.  Anterior Regions of Monkey Parietal Cortex Process Visual 3D Shape , 2007, Neuron.

[56]  Nikos K. Logothetis,et al.  Three-Dimensional Shape Representation in Monkey Cortex , 2002, Neuron.