A Bayesian approach to the stereo correspondence problem

In this thesis we present a method for reconstructing the three-dimensional scene geometry (i.e. depth, surface orientation, discontinuities in depth, and surface creases) from a pair of stereo images. We start by developing a treatment of the half-occluded regions--regions in a stereo pair representing points in space visible to only one of the two cameras or eyes. Previous approaches to stereo either ignored these unmatchable points or attempted to eliminate them in a second pass. Our algorithm incorporates them from the start as a strong clue to determining depth discontinuities. For our algorithm, we develop an energy functional as a Bayesian maximum a posteriori (MAP) estimator of the quantities in the scene geometry. We focus primarily on the derivation of the prior, showing there is a sizable class of energy functionals that implicitly assumes prior models that are constructed from the sums of Brownian motion processes and compound Poisson processes. We argue that the prior assumptions which produce this class of energy functionals accurately model the scene geometry for stereo images. In addition, we develop a Bayesian feedback method for incorporating global interaction into our prior model. Since most stereo scenes contain either background continuation (large background surfaces continuing behind smaller foreground surfaces) or transparency continuation (small opaque patches on a transparent surface), highly non-local interactions are often present in the scene geometry. The local prior models developed in the middle stage of this thesis are unable to capture the probabilistic subtleties of global 3-D structures. Therefore, we develop a hybridized prior which balances the local properties of the scene geometry with the global properties. Finally, we present experimental results demonstrating the effectiveness of these approaches.