Occlusion filling in stereo: Theory and experiments

Disparity maps, occlusions, and occlusion filling results on the Middlebury College test images Map, Venus, and Tsukuba (from top to bottom). Occlusions are filled using SLS linear interpolation model.Display Omitted Highlights? Comprehensive study of different occlusions and their origins. ? Discussion of ambiguities in occlusion filling. ? Achieve higher accuracy in occlusion filling applying color homogeneity. ? Probabilistic definition of homogeneity to overcome user-defined thresholds. A number of stereo matching algorithms have been developed in the last few years, which also have successfully detected occlusions in stereo images. These algorithms typically fall short of a systematic study of occlusions; they predominantly emphasize matching and regard occlusion filling as a secondary operation. Filling occlusions, however, is useful in many applications such as image-based rendering where 3D models are desired to be as complete as possible. In this paper, we study occlusions in a systematic way and propose two algorithms to fill occlusions reliably by applying statistical modeling, visibility constraints, and scene constraints. We introduce a probabilistic, model-based filling order of the occluded points to maintain consistency in filling. Furthermore, we show how an ambiguity in the interpolation of the disparity value of an occluded point can safely be avoided using color homogeneity when the point's neighborhood consists of multiple scene surfaces. We perform a comparative study and show that statistically, the new algorithms deliver good quality results compared to existing algorithms.