Computational models of binocular vision

We have developed a new stereo algorithm that goes beyond edge-based and area-based approaches by using an image description that is richer than attributes of edges or pixel-brightness values. The algorithm also incorporates several stereo cues that have previously been neglected in the computer vision literature. Any stereo algorithm must have at its core a basic and robust mechanism for establishing correspondence between two images. We use linear spatial filters at a range of scales and orientations that are similar to those known to exist in the visual cortex. Filter responses at a given point characterize the local image patch, providing a more concise representation than traditional correlation-based techniques. A simple similarity measure, motivated by robust statistics, serves as a basis for establishing correspondence between image patches in two views. From initial horizontal and vertical disparity estimates, additional information can be recovered about a scene. The viewing geometry, along with the locations of depth discontinuities and occluded regions, all can be recovered and used to iteratively refine the disparity estimates. Orientation and spatial frequency disparities can be related geometrically to the slant and tilt of surfaces in a scene. These cues have traditionally been regarded as noise when determining point correspondences. We have developed two new algorithms for recovering local surface orientation based on orientation and spatial frequency disparities: (1) a least squares algorithm based on measurements of corresponding line segments, and (2) a method that makes use of filter responses directly. The second method has also been shown to account for some previously unexplained psychophysical observations by Tyler and Sutter, and von der Heydt. In primary visual cortex of both macaque monkey and cat, inputs from the two eyes are segregated into alternating zones known as ocular dominance bands. If the striking differences in the overall arrangements of these inputs reflect important differences in the basic rules governing cortical ocular dominance, then this poses a problem for attempts to formulate general principles of visual cortical organization. We have formulated and tested a single computational model that accurately predicts the quite dissimilar ocular dominance pattern in cat and monkey. This model also generalizes to predict the ocular dominance pattern in three-eyed frogs, supporting the notion that the overall pattern of ocular dominance is governed by a common set of rules.