Disparity Estimation by Pooling Evidence From Energy Neurons

In this paper, we propose an algorithm for disparity estimation from disparity energy neurons that seeks to maintain simplicity and biological plausibility, while also being based upon a formulation that enables us to interpret the model outputs probabilistically. We use the Bayes factor from statistical hypothesis testing to show that, in contradiction to the implicit assumption of many previously proposed biologically plausible models, a larger response from a disparity energy neuron does not imply more evidence for the hypothesis that the input disparity is close to the preferred disparity of the neuron. However, we find that the normalized response can be interpreted as evidence, and that information from different orientation channels can be combined by pooling the normalized responses. Based on this insight, we propose an algorithm for disparity estimation constructed out of biologically plausible operations. Our experimental results on real stereograms show that the algorithm outperforms a previously proposed coarse-to-fine model. In addition, because its outputs can be interpreted probabilistically, the model also enables us to identify occluded pixels or pixels with incorrect disparity estimates.

[1]  J. H. van Hateren,et al.  Modelling the Power Spectra of Natural Images: Statistics and Information , 1996, Vision Research.

[2]  Hermann Wagner,et al.  Visual depth encoding in populations of neurons with localized receptive fields , 2002, Biological Cybernetics.

[3]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[4]  Bruce G Cumming,et al.  Ocular dominance predicts neither strength nor class of disparity selectivity with random-dot stimuli in primate V1. , 2004, Journal of neurophysiology.

[5]  Ning Qian,et al.  Physiological computation of binocular disparity , 1997, Vision Research.

[6]  Ning Qian,et al.  Binocular Receptive Field Models, Disparity Tuning, and Characteristic Disparity , 1996, Neural Computation.

[7]  Bertram E. Shi,et al.  Normalization Enables Robust Validation of Disparity Estimates from Neural Populations , 2008, Neural Computation.

[8]  Bruce G Cumming,et al.  Sensors for impossible stimuli may solve the stereo correspondence problem , 2007, Nature Neuroscience.

[9]  Bertram E. Shi,et al.  Estimating disparity with confidence from energy neurons , 2007, NIPS.

[10]  Ning Qian,et al.  A Coarse-to-Fine Disparity Energy Model with Both Phase-Shift and Position-Shift Receptive Field Mechanisms , 2004, Neural Computation.

[11]  Heiko Hirschmüller,et al.  Evaluation of Cost Functions for Stereo Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  D. G. Albrecht,et al.  Motion selectivity and the contrast-response function of simple cells in the visual cortex , 1991, Visual Neuroscience.

[13]  I. Ohzawa,et al.  Neural mechanisms for processing binocular information I. Simple cells. , 1999, Journal of neurophysiology.

[14]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[15]  Ning Qian,et al.  Computing Stereo Disparity and Motion with Known Binocular Cell Properties , 1994, Neural Computation.

[16]  Jenny C. A. Read,et al.  A Bayesian Approach to the Stereo Correspondence Problem , 2002, Neural Computation.

[17]  Jeffrey A. Sloan,et al.  Spatial frequency analysis of the visual environment: Anisotropy and the carpentered environment hypothesis , 1978, Vision Research.

[18]  L. M. M.-T..  Theory of Probability , 1929, Nature.

[19]  Ayoub Al-Hamadi,et al.  Dense depth map reconstruction by phase difference-based algorithm under influence of perspective distortion , 2006 .

[20]  Gregory C. DeAngelis,et al.  Depth is encoded in the visual cortex by a specialized receptive field structure , 1991, Nature.

[21]  Christopher Joseph Pal,et al.  Learning Conditional Random Fields for Stereo , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Richard D. Deveaux,et al.  Applied Smoothing Techniques for Data Analysis , 1999, Technometrics.

[23]  D Marr,et al.  A computational theory of human stereo vision. , 1979, Proceedings of the Royal Society of London. Series B, Biological sciences.

[24]  Jonathan D. Victor,et al.  Reading a population code: a multi-scale neural model for representing binocular disparity , 2003, Vision Research.

[25]  S. Lehky,et al.  Neural model of stereoacuity and depth interpolation based on a distributed representation of stereo disparity [published erratum appears in J Neurosci 1991 Mar;11(3):following Table of Contents] , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[26]  I. Ohzawa,et al.  Stereoscopic depth discrimination in the visual cortex: neurons ideally suited as disparity detectors. , 1990, Science.

[27]  Jenny C. A. Read.  A Bayesian model of stereopsis depth and motion direction discrimination , 2002, Biological Cybernetics.

[28]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[29]  David J. Fleet,et al.  Neural encoding of binocular disparity: Energy models, position shifts and phase shifts , 1996, Vision Research.

[30]  S. El-Etriby,et al.  Dense Stereo Correspondence with Slanted Surface using Phase-based Algorithm , 2007, 2007 IEEE International Symposium on Industrial Electronics.

[31]  J. Movshon,et al.  Linearity and Normalization in Simple Cells of the Macaque Primary Visual Cortex , 1997, The Journal of Neuroscience.

[32]  A. M. Rohaly,et al.  Disparity averaging across spatial scales , 1994, Vision Research.

[33]  G. Poggio,et al.  Binocular interaction and depth sensitivity in striate and prestriate cortex of behaving rhesus monkey. , 1977, Journal of neurophysiology.

[34]  D. Heeger.  Normalization of cell responses in cat striate cortex , 1992, Visual Neuroscience.

[35]  I. Ohzawa,et al.  Neural mechanisms for encoding binocular disparity: receptive field position versus phase. , 1999, Journal of neurophysiology.

[36]  Peter Tino,et al.  IEEE Transactions on Neural Networks , 2009 .

[37]  I. Ohzawa,et al.  Neural mechanisms for processing binocular information II. Complex cells. , 1999, Journal of neurophysiology.

[38]  D. Pollen,et al.  Spatial and temporal frequency selectivity of neurones in visual cortical areas V1 and V2 of the macaque monkey. , 1985, The Journal of physiology.