Light-field flow: A subpixel-accuracy depth flow estimation with geometric occlusion model from a single light-field image

Light-field cameras capture not only 2D images, but also the angles of the incoming light. These additional light angles bring the benefit of getting a sub-aperture image array from a single light-field image. Inspired by the traditional optical flow with occlusion detection, this paper focuses on the correlation analysis and the occlusion modeling for the sub-aperture array, and unifies them into a light-field flow framework. The main challenges faced are subpixel displacements and occlusion handling among the sub-aperture images. We build a light-field flow for joint depth estimation and occlusion detection, and develop a geometric occlusion model. More specifically, we firstly estimate subpixel-accuracy optical flows from each two sub-aperture images by the phase shift theorem, then a forward-backward consistency checking is adopted to detect the occluded regions. According to the geometric complementary character of occlusion in a light-field image, an occlusion filling strategy is proposed to refine depth estimation in the occluded regions. Experimental results on the synthetic scenes and Lytro Illum camera data both demonstrate the effectiveness and robustness of our method which has excellent performance in handling occlusions.