Epipolar Plane Image Refocusing for Improved Depth Estimation and Occlusion Handling

In contrast to traditional imaging, the higher dimensionality of a light field offers directional information about the captured intensity. This information can be leveraged to estimate the disparity of 3D points in the captured scene. A recent approach to estimate disparities analyzes the structure tensor and evaluates the orientation on epipolar plane images (EPIs). While the resulting disparity maps are generally satisfying, the allowed disparity range is small and occlusion boundaries can become smeared and noisy. In this paper, we first introduce an approach to extend the total allowed disparity range. This allows for example the investigation of camera setups with a larger baseline, like in the Middlebury 3D light fields. Second, we introduce a method to handle the difficulties arising at boundaries between foreand background objects to achieve sharper edge transitions.

[1]  Jan-Michael Frahm,et al.  Real-Time Plane-Sweeping Stereo with Multiple Sweeping Directions , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Kiyoharu Aizawa,et al.  Reconstructing Dense Light Field From Array of Multifocus Images for Novel View Synthesis , 2007, IEEE Transactions on Image Processing.

[3]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[4]  Sven Wanner,et al.  Globally Consistent Multi-label Assignment on the Ray Space of 4D Light Fields , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  J. Bigun,et al.  Optimal Orientation Detection of Linear Symmetry , 1987, ICCV 1987.

[6]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[7]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.

[8]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[9]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[10]  Sven Wanner,et al.  Variational Light Field Analysis for Disparity Estimation and Super-Resolution , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Sven Wanner,et al.  Globally consistent depth labeling of 4D light fields , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  M. Landy,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[13]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[14]  Marc Levoy,et al.  Synthetic Aperture Focusing using a Shear-Warp Factorization of the Viewing Transform , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[15]  Jan-Michael Frahm,et al.  Fast Global Labeling for Real-Time Stereo Using Multiple Plane Sweeps , 2008, VMV.

[16]  Heiko Hirschmüller,et al.  Evaluation of Cost Functions for Stereo Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Marc Levoy,et al.  Using plane + parallax for calibrating dense camera arrays , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[18]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[19]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[20]  Christopher Joseph Pal,et al.  Learning Conditional Random Fields for Stereo , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Daniel Cremers,et al.  Global Solutions of Variational Models with Convex Regularization , 2010, SIAM J. Imaging Sci..

[22]  Andrew Lumsdaine,et al.  Focused plenoptic camera and rendering , 2010, J. Electronic Imaging.

[23]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[24]  Richard Szeliski,et al.  Extracting layers and analyzing their specular properties using epipolar-plane-image analysis , 2005, Comput. Vis. Image Underst..

[25]  Marc Levoy,et al.  Reconstructing Occluded Surfaces Using Synthetic Apertures: Stereo, Focus and Robust Measures , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).