Cross-matching: a modified cross-correlation underlying threshold energy model and match-based depth perception

Three-dimensional visual perception requires correct matching of images projected to the left and right eyes. The matching process is faced with an ambiguity: part of one eye's image can be matched to multiple parts of the other eye's image. This stereo correspondence problem is complicated for random-dot stereograms (RDSs), because dots with an identical appearance produce numerous potential matches. Despite such complexity, human subjects can perceive a coherent depth structure. A coherent solution to the correspondence problem does not exist for anticorrelated RDSs (aRDSs), in which luminance contrast is reversed in one eye. Neurons in the visual cortex reduce disparity selectivity for aRDSs progressively along the visual processing hierarchy. A disparity-energy model followed by threshold nonlinearity (threshold energy model) can account for this reduction, providing a possible mechanism for the neural matching process. However, the essential computation underlying the threshold energy model is not clear. Here, we propose that a nonlinear modification of cross-correlation, which we term “cross-matching,” represents the essence of the threshold energy model. We placed half-wave rectification within the cross-correlation of the left-eye and right-eye images. The disparity tuning derived from cross-matching was attenuated for aRDSs. We simulated a psychometric curve as a function of graded anticorrelation (graded mixture of aRDS and normal RDS); this simulated curve reproduced the match-based psychometric function observed in human near/far discrimination. The dot density was 25% for both simulation and observation. We predicted that as the dot density increased, the performance for aRDSs should decrease below chance (i.e., reversed depth), and the level of anticorrelation that nullifies depth perception should also decrease. We suggest that cross-matching serves as a simple computation underlying the match-based disparity signals in stereoscopic depth perception.

[1]  Ning Qian,et al.  Physiological computation of binocular disparity , 1997, Vision Research.

[2]  Martin S Banks,et al.  Limits of stereopsis explained by local cross-correlation. , 2009, Journal of vision.

[3]  Richard A Eagle,et al.  Weighted directional energy model of human stereo correspondence , 2000, Vision Research.

[4]  R. Freeman,et al.  Stereoscopic depth processing in the visual cortex: a coarse-to-fine mechanism , 2003, Nature Neuroscience.

[5]  Takahiro Doi,et al.  Matching and correlation computations in stereoscopic depth perception. , 2011, Journal of vision.

[6]  J. Pokorny.  Foundations of Cyclopean Perception , 1972 .

[7]  Takahiro Doi,et al.  Neural Activity in Cortical Area V4 Underlies Fine Disparity Discrimination , 2012, The Journal of Neuroscience.

[8]  Takahiro Doi,et al.  Disparity-tuning characteristics of neuronal responses to dynamic random-dot stereograms in macaque visual area V4. , 2005, Journal of neurophysiology.

[9]  Bruce G. Cumming,et al.  Adaptation to Natural Binocular Disparities in Primate V1 Explained by a Generalized Energy Model , 2008, Neuron.

[10]  Bruce G Cumming,et al.  Mechanisms Underlying the Transformation of Disparity Signals from V1 to V2 in the Macaque , 2008, The Journal of Neuroscience.

[11]  I. Ohzawa,et al.  Neural mechanisms for processing binocular information I. Simple cells. , 1999, Journal of neurophysiology.

[12]  H. Wagner,et al.  Hierarchical Processing of Horizontal Disparity Information in the Visual Forebrain of Behaving Owls , 2001, The Journal of Neuroscience.

[13]  Ichiro Fujita,et al.  Disparity-energy signals in perceived stereoscopic depth. , 2008, Journal of vision.

[14]  J. A. Pruszynski,et al.  Neural correlates , 2023 .

[15]  D Marr,et al.  Cooperative computation of stereo disparity. , 1976, Science.

[16]  David J. Fleet,et al.  Neural encoding of binocular disparity: Energy models, position shifts and phase shifts , 1996, Vision Research.

[17]  Paul B. Hibbard,et al.  Depth Perception Not Found in Human Observers for Static or Dynamic Anti-Correlated Random Dot Stereograms , 2014, PloS one.

[18]  Bruce G Cumming,et al.  Delayed suppression shapes disparity selective responses in monkey V1. , 2014, Journal of neurophysiology.

[19]  Bruce G Cumming,et al.  A simple model accounts for the response of disparity-tuned V1 neurons to anticorrelated images , 2002, Visual Neuroscience.

[20]  Tai Sing Lee,et al.  Cooperative and Competitive Interactions Facilitate Stereo Computations in Macaque Primary Visual Cortex , 2009, The Journal of Neuroscience.

[21]  D Marr,et al.  A computational theory of human stereo vision. , 1979, Proceedings of the Royal Society of London. Series B, Biological sciences.

[22]  A. Parker,et al.  Comparing perceptual signals of single V5/MT neurons in two binocular depth tasks. , 2004, Journal of neurophysiology.

[23]  Richard A Eagle,et al.  Reversed stereo depth and motion direction with anti-correlated stimuli , 2000, Vision Research.

[24]  I. Ohzawa,et al.  Spatiotemporal organization of simple-cell receptive fields in the cat's striate cortex. I. General characteristics and postnatal development. , 1993, Journal of neurophysiology.

[25]  I. Fujita,et al.  Rejection of False Matches for Binocular Correspondence in Macaque Visual Cortical Area V4 , 2004, The Journal of Neuroscience.

[26]  Bruce G Cumming,et al.  Sensors for impossible stimuli may solve the stereo correspondence problem , 2007, Nature Neuroscience.

[27]  A. Parker,et al.  Range and mechanism of encoding of horizontal disparity in macaque V1. , 2002, Journal of neurophysiology.

[28]  G. DeAngelis,et al.  Contribution of Area MT to Stereoscopic Depth Perception Choice-Related Response Modulations Reflect Task Strategy , 2004, Neuron.

[29]  G. Orban,et al.  At Least at the Level of Inferior Temporal Cortex, the Stereo Correspondence Problem Is Solved , 2003, Neuron.

[30]  L. Cormack,et al.  Interocular correlation, luminance contrast and cyclopean processing , 1991, Vision Research.

[31]  A. Parker,et al.  A specialization for relative disparity in V2 , 2002, Nature Neuroscience.

[32]  H. Wagner,et al.  A threshold explains modulation of neural responses to opposite-contrast stereograms , 2001, Neuroreport.

[33]  Ichiro Fujita,et al.  Neural Correlates of Fine Depth Discrimination in Monkey Inferior Temporal Cortex , 2005, The Journal of Neuroscience.

[34]  M. Shadlen,et al.  Microstimulation of visual cortex affects the speed of perceptual decisions , 2003, Nature Neuroscience.

[35]  B. G. Cumming,et al.  Responses of primary visual cortical neurons to binocular disparity without depth perception , 1997, Nature.

[36]  I. Ohzawa,et al.  Stereoscopic depth discrimination in the visual cortex: neurons ideally suited as disparity detectors. , 1990, Science.

[37]  Takahiro Doi,et al.  Temporal channels and disparity representations in stereoscopic depth perception. , 2013, Journal of vision.

[38]  Peter Janssen,et al.  Selectivity for three-dimensional contours and surfaces in the anterior intraparietal area. , 2012, Journal of neurophysiology.

[39]  B. Cumming,et al.  Suppressive Mechanisms in Monkey V1 Help to Solve the Stereo Correspondence Problem , 2011, The Journal of Neuroscience.

[40]  Z. Kourtzi,et al.  Multivoxel Pattern Selectivity for Perceptually Relevant Binocular Disparities in the Human Brain , 2008, The Journal of Neuroscience.

[41]  J. Movshon,et al.  A computational analysis of the relationship between neuronal and behavioral responses to visual motion , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[42]  Ichiro Fujita,et al.  Spatial frequency integration for binocular correspondence in macaque area V4. , 2008, Journal of neurophysiology.

[43]  Ning Qian,et al.  A Coarse-to-Fine Disparity Energy Model with Both Phase-Shift and Position-Shift Receptive Field Mechanisms , 2004, Neural Computation.