Spatial Stereoresolution for Depth Corrugations May Be Set in Primary Visual Cortex

Stereo depth perception has recently been modelled based on local cross-correlation between the left and the right eye’s images. This model, which is based on the known physiology of primary visual cortex (V1), has successfully explained many aspects of stereo vision. In particular, it has explained the low spatial stereoresolution for sinusoidal depth corrugations [1,2], suggesting that the limit on stereoresolution may be set in V1. In accordance with the properties of V1 neurons, the disparity detectors used in this model are tuned to locally uniform patches of disparity. Consequently, the model responds better to high amplitude square-wave corrugations than to high amplitude sine-waves, because the square-waves are locally flat while the sinusoidal corrugations are slanted almost everywhere and this slant is particularly large at large amplitudes. The model therefore predicts better performance at detecting square-wave than sine-wave disparity corrugations at high amplitudes. However, in contradiction with this prediction of the model we have recently shown that humans perform no better at detecting square-waves than sine-waves even at high amplitudes [3]. This failure of the model raised the question of whether stereoresolution is not set in V1 but at some later stage of cortical processing, for example involving neurons tuned to slant or curvature or whether a modified version of the model, incorporating more of the known physiology, may explain the new results with square-waves. We have tested a modified version of the local cross-correlation model which, based on psychophysical and physiological evidence that larger disparities are detected by neurons with larger receptive fields (a size-disparity correlation), uses larger windows to detect larger disparities. We show that the performance of this modified model is consistent with the human results, confirming that stereoresolution may indeed be limited by V1 receptive field sizes.

[1]  Bruce G Cumming,et al.  Does depth perception require vertical-disparity detectors? , 2006, Journal of vision.

[2]  H. Sakata,et al.  Neural representation of three-dimensional features of manipulation objects with stereopsis , 1999, Experimental Brain Research.

[3]  Mark F. Bradshaw,et al.  Sensitivity to horizontal and vertical corrugations defined by binocular disparity , 1999, Vision Research.

[4]  I. Ohzawa,et al.  Stereoscopic depth discrimination in the visual cortex: neurons ideally suited as disparity detectors. , 1990, Science.

[5]  Takeo Kanade,et al.  A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Jenny C. A. Read.  A Bayesian model of stereopsis depth and motion direction discrimination , 2002, Biological Cybernetics.

[7]  Ning Qian,et al.  Computing Stereo Disparity and Motion with Known Binocular Cell Properties , 1994, Neural Computation.

[8]  Ignacio Serrano-Pedraza,et al.  Multiple channels for horizontal, but only one for vertical corrugations? A new look at the stereo anisotropy. , 2010, Journal of vision.

[9]  Preeti Verghese,et al.  Stereo transparency and the disparity gradient limit , 2002, Vision Research.

[10]  G Westheimer,et al.  Editorial: Visual acuity and hyperacuity. , 1975, Investigative ophthalmology.

[11]  Marsha Jo Hannah,et al.  Computer matching of areas in stereo images. , 1974 .

[12]  A. Parker,et al.  Range and mechanism of encoding of horizontal disparity in macaque V1. , 2002, Journal of neurophysiology.

[13]  Bruce G Cumming,et al.  Sensors for impossible stimuli may solve the stereo correspondence problem , 2007, Nature Neuroscience.

[14]  Bruce G. Cumming,et al.  Understanding the Cortical Specialization for Horizontal Disparity , 2004, Neural Computation.

[15]  B. Julesz,et al.  A disparity gradient limit for binocular fusion. , 1980, Science.

[16]  Ning Qian,et al.  Physiological computation of binocular disparity , 1997, Vision Research.

[17]  Inna Tsirlin,et al.  Stereoscopic transparency: constraints on the perception of multiple surfaces. , 2008, Journal of vision.

[18]  Hermann Wagner,et al.  Disparity sensitivity in man and owl: Psychophysical evidence for equivalent perception of shape-from-stereo. , 2011, Journal of vision.

[19]  Julie M. Harris,et al.  Fine-scale processing in human binocular stereopsis. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[20]  R. Andersen,et al.  Response of MSTd neurons to simulated 3D orientation of rotating planes. , 2002, Journal of neurophysiology.

[21]  Jonathan D. Victor,et al.  Reading a population code: a multi-scale neural model for representing binocular disparity , 2003, Vision Research.

[22]  A. Parker.  Binocular depth perception and the cerebral cortex , 2007, Nature Reviews Neuroscience.

[23]  C. Schor,et al.  Disparity range for local stereopsis as a function of luminance spatial frequency , 1983, Vision Research.

[24]  David J. Fleet,et al.  Phase-based disparity measurement , 1991, CVGIP Image Underst..

[25]  C. WILLIAM TYLER,et al.  Depth perception in disparity gratings , 1974, Nature.

[26]  R. Andersen,et al.  Intention-related activity in the posterior parietal cortex: a review , 2000, Vision Research.

[27]  J. Pokorny.  Foundations of Cyclopean Perception , 1972 .

[28]  C. Tyler.  Spatial organization of binocular disparity sensitivity , 1975, Vision Research.

[29]  David J. Fleet,et al.  Neural encoding of binocular disparity: Energy models, position shifts and phase shifts , 1996, Vision Research.

[30]  Hong Zhou,et al.  Representation of stereoscopic edges in monkey visual cortex , 2000, Vision Research.

[31]  Jenny C. A. Read.  Vertical Binocular Disparity is Encoded Implicitly within a Model Neuronal Population Tuned to Horizontal Disparity and Orientation , 2010, PLoS Comput. Biol..

[32]  RussLL L. Ds Vnlos,et al.  SPATIAL FREQUENCY SELECTIVITY OF CELLS IN MACAQUE VISUAL CORTEX , 2022 .

[33]  H. Smallman,et al.  Size-disparity correlation in stereopsis at contrast threshold. , 1994, Journal of the Optical Society of America. A, Optics, image science, and vision.

[34]  Jerry D. Nguyenkim,et al.  Disparity-Based Coding of Three-Dimensional Surface Orientation by Macaque Middle Temporal Neurons , 2003, The Journal of Neuroscience.

[35]  B. G. Cumming,et al.  An unexpected specialization for horizontal disparity in primate primary visual cortex , 2002, Nature.

[36]  R Vogels,et al.  Macaque inferior temporal neurons are selective for disparity-defined three-dimensional shapes. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[37]  Uwe Franke,et al.  Performance Evaluation of Stereo Algorithms for Automotive Applications , 2009, ICVS.

[38]  Bruce G. Cumming,et al.  A Simple Account of Cyclopean Edge Responses in Macaque V2 , 2006, The Journal of Neuroscience.

[39]  A. Parker,et al.  Receptive Field Size in V1 Neurons Limits Acuity for Perceiving Disparity Modulation , 2004, The Journal of Neuroscience.

[40]  Preeti Verghese,et al.  What is the depth of a sinusoidal grating? , 2004, Journal of vision.

[41]  C. Tyler.  Stereoscopic Vision: Cortical Limitations and a Disparity Scaling Effect , 1973, Science.

[42]  Martin S Banks,et al.  Limits of stereopsis explained by local cross-correlation. , 2009, Journal of vision.

[43]  O. Schade.  Optical and photoelectric analog of the eye. , 1956, Journal of the Optical Society of America.

[44]  Ning Qian,et al.  Relationship Between Phase and Energy Methods for Disparity Computation , 2000, Neural Computation.

[45]  Jenny C A Read,et al.  Detectability of sine- versus square-wave disparity gratings: A challenge for current models of depth perception. , 2010, Journal of vision.

[46]  A. Parker,et al.  Binocular Neurons in V1 of Awake Monkeys Are Selective for Absolute, Not Relative, Disparity , 1999, The Journal of Neuroscience.

[47]  Gregory C. DeAngelis,et al.  Depth is encoded in the visual cortex by a specialized receptive field structure , 1991, Nature.

[48]  Keith Langley,et al.  Surface orientation, modulation frequency and the detection and perception of depth defined by binocular disparity and motion parallax , 2006, Vision Research.

[49]  M. Landy,et al.  Why Is Spatial Stereoresolution So Low? , 2004, The Journal of Neuroscience.

[50]  J. J. KULIKOWSKI,et al.  Limit of single vision in stereopsis depends on contour sharpness , 1978, Nature.

[51]  L. Cormack,et al.  Interocular correlation, luminance contrast and cyclopean processing , 1991, Vision Research.

[52]  John J. B. Allen,et al.  Neurophysiological evidence for the influence of past experience on figure-ground perception. , 2010, Journal of vision.