Occlusion-Model Guided Antiocclusion Depth Estimation in Light Field

Occlusion is one of the most challenging problems in depth estimation. Previous work has modeled the single-occluder occlusion in light field and achieves good performances, however it is still difficult to obtain accurate depth for multioccluder occlusion. In this paper, we explore the complete occlusion model in light field and derive the occluder-consistency between the spatial and angular spaces, which is used as a guidance to select unoccluded views for each candidate occlusion point. Then, an antiocclusion energy function is built to regularize the depth map. Experimental results on both synthetic and real light-field datasets have demonstrated the advantages of the proposed algorithm compared with state-of-the-art algorithms of light-field depth estimation, especially in multioccluder cases.

[1]  P. Hanrahan,et al.  Digital light field photography , 2006 .

[2]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Adrien Bousseau,et al.  How do people edit light fields? , 2014, ACM Trans. Graph..

[4]  Yael Pritch,et al.  Scene reconstruction from high spatio-angular resolution light fields , 2013, ACM Trans. Graph..

[5]  Qionghai Dai,et al.  Light field from micro-baseline image pair , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[7]  Marc Levoy,et al.  Reconstructing Occluded Surfaces Using Synthetic Apertures: Stereo, Focus and Robust Measures , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[8]  Vladimir Kolmogorov,et al.  Multi-camera Scene Reconstruction via Graph Cuts , 2002, ECCV.

[9]  Frédo Durand,et al.  Linear view synthesis using a dimensionality gap light field prior , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[12]  In-So Kweon,et al.  Accurate depth map estimation from a lenslet light field camera , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Sven Wanner,et al.  Variational Light Field Analysis for Disparity Estimation and Super-Resolution , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Zhaolin Xiao,et al.  Aliasing Detection and Reduction in Plenoptic Imaging , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Alexei A. Efros,et al.  Occlusion-Aware Depth Estimation Using Light-Field Cameras , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[16]  Ting-Chun Wang,et al.  Learning-based view synthesis for light field cameras , 2016, ACM Trans. Graph..

[17]  Andreas Geiger,et al.  Object scene flow for autonomous vehicles , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Abhilash Sunder Raj.  Light-Field Database Creation and Depth Estimation , 2016 .

[19]  Pushmeet Kohli,et al.  Surface stereo with soft segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[21]  Zhan Yu,et al.  Line Assisted Light Field Triangulation and Stereo Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[22]  Yasuhiro Mukaigawa,et al.  4D light field segmentation with spatial and angular consistencies , 2016, 2016 IEEE International Conference on Computational Photography (ICCP).

[23]  Haibin Ling,et al.  Saliency Detection on Light Field , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Ian Reid,et al.  Global stereo reconstruction under second order smoothness priors , 2008, CVPR.

[25]  Qionghai Dai,et al.  Light Field Image Processing: An Overview , 2017, IEEE Journal of Selected Topics in Signal Processing.

[26]  VekslerOlga,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001 .

[27]  Y. Anzai,et al.  Pattern Recognition & Machine Learning , 2016 .

[28]  Alexei A. Efros,et al.  SVBRDF-Invariant Shape and Reflectance Estimation from Light-Field Cameras , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Sven Wanner,et al.  Variational Light Field Analysis for Disparity Estimation and Super-Resolution. , 2013, IEEE transactions on pattern analysis and machine intelligence.

[30]  Sven Wanner,et al.  Globally consistent depth labeling of 4D light fields , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Alexei A. Efros,et al.  Depth Estimation with Occlusion Modeling Using Light-Field Cameras , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[33]  Qing Wang,et al.  Aliasing Detection and Reduction Scheme on Angularly Undersampled Light Fields , 2017, IEEE Transactions on Image Processing.

[34]  Zhaolin Xiao,et al.  Seeing Beyond Foreground Occlusion: A Joint Framework for SAP-Based Scene Depth and Appearance Reconstruction , 2017, IEEE Journal of Selected Topics in Signal Processing.

[35]  Kiriakos N. Kutulakos,et al.  What does an aberrated photo tell us about the lens and the scene? , 2013, IEEE International Conference on Computational Photography (ICCP).

[36]  Zhan Yu,et al.  Light Field Stereo Matching Using Bilateral Statistics of Surface Cameras , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Jitendra Malik,et al.  Depth from Combining Defocus and Correspondence Using Light-Field Cameras , 2013, 2013 IEEE International Conference on Computer Vision.

[38]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[39]  Andrew W. Fitzgibbon,et al.  Global stereo reconstruction under second order smoothness priors , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Vladimir Kolmogorov,et al.  Optimizing Binary MRFs via Extended Roof Duality , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Alexei A. Efros,et al.  SVBRDF-Invariant Shape and Reflectance Estimation from a Light-Field Camera , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Sven Wanner,et al.  Datasets and Benchmarks for Densely Sampled 4D Light Fields , 2013, VMV.

[43]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Meng Wang,et al.  Light field saliency vs. 2D saliency: A comparative study , 2015, Neurocomputing.