An Effective Occlusion Edge Prediction Method in Light Field Depth Estimation

Occlusion is one of the most complicated and difficult problem to deal with in light field depth estimation. Previous methods usually use edge detection of central view as the basis for subsequent depth estimation. However, few algorithms take the correlation between light field sub-aperture images into consideration, which makes the incorrect depth estimation in the occlusion areas and object boundaries. In this paper, based on the refocusing theory, we explore three occlusion prediction methods and derive model to make full use of the correlation between different sub-aperture to handle the occlusions in depth estimation, the obtained occlusion map is used as a guidance for the depth estimation. Markov Random Field is built to regularize the depth map and smoothing filtering, which is used at the background area of the non-occlusion area to process noise in order to improve estimation results. Experimental results on the real light-field data sets show that the proposed algorithm can greatly improve the depth estimation results compared with state-of-the-art algorithms, especially in complex occlusion areas.

[1]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  L. Tian,et al.  3D intensity and phase imaging from light field measurements in an LED array microscope , 2015 .

[3]  Sven Wanner,et al.  Globally consistent depth labeling of 4D light fields , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Sanja Fidler,et al.  3D Graph Neural Networks for RGBD Semantic Segmentation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[5]  Chao Li,et al.  Robust depth estimation for light field via spinning parallelogram operator , 2016, Comput. Vis. Image Underst..

[6]  Can Chen,et al.  Depth Recovery from Light Field Using Focal Stack Symmetry , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[7]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Ze-Nian Li,et al.  Continuous depth map reconstruction from light fields , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[9]  In-So Kweon,et al.  Accurate depth map estimation from a lenslet light field camera , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  In Kyu Park,et al.  Robust Light Field Depth Estimation Using Occlusion-Noise Aware Data Costs , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Nasir M. Rajpoot,et al.  Photo-consistency and multiresolution methods for light field disparity estimation , 2005 .

[12]  Zhan Yu,et al.  Light Field Stereo Matching Using Bilateral Statistics of Surface Cameras , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Jitendra Malik,et al.  Depth from Combining Defocus and Correspondence Using Light-Field Cameras , 2013, 2013 IEEE International Conference on Computer Vision.

[14]  Atsushi Shimada,et al.  Light Field Distortion Feature for Transparent Object Recognition , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Qing Wang,et al.  Occlusion-Model Guided Antiocclusion Depth Estimation in Light Field , 2016, IEEE Journal of Selected Topics in Signal Processing.

[16]  Thomas Pock,et al.  Variational Shape from Light Field , 2013, EMMCVPR.

[17]  Zhan Yu,et al.  Line Assisted Light Field Triangulation and Stereo Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[18]  David A. Forsyth,et al.  Editorial: State of the Journal , 2014, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Sven Wanner,et al.  Variational Light Field Analysis for Disparity Estimation and Super-Resolution , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Sven Wanner,et al.  Datasets and Benchmarks for Densely Sampled 4D Light Fields , 2013, VMV.

[21]  Alexei A. Efros,et al.  Occlusion-Aware Depth Estimation Using Light-Field Cameras , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Bastian Goldlücke,et al.  A Dataset and Evaluation Methodology for Depth Estimation on 4D Light Fields , 2016, ACCV.

[23]  Meng Wang,et al.  Saliency Detection with a Deeper Investigation of Light Field , 2015, IJCAI.

[24]  Hao Wang,et al.  A Method for Handling Multi-Occlusion in Depth Estimation of Light Field , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[25]  Yo-Sung Ho,et al.  Initial depth estimation using adaptive window size with light field image , 2018, 2018 International Workshop on Advanced Image Technology (IWAIT).

[26]  Wei An,et al.  Disparity Estimation for Camera Arrays Using Reliability Guided Disparity Propagation , 2018, IEEE Access.