Fast Depth Estimation for Light Field Cameras

Fast depth estimation for light field images is an important task for multiple applications such as image-based rendering and refocusing. Most previous approaches to light field depth estimation involve high computational costs. Therefore, in this study, we propose a fast depth estimation method based on multi-view stereo matching for light field images. Similar to other conventional methods, our method consists of initial depth estimation and refinement. For the initial estimation, we use a one-bit feature for each pixel and calculate matching costs by summing all combinations of viewpoints with a fast algorithm. To reduce computational time, we introduce an offline viewpoint selection strategy and cost volume interpolation. Our refinement process solves the minimization problem in which the objective function consists of $\ell _{1}$ data and smoothness terms. Although this problem can be solved via a graph cuts algorithm, it is computationally expensive; therefore, we propose an approximate solver based on a fast-weighted median filter. Experiments on synthetic and real-world data show that our method achieves competitive accuracy with the shortest computational time of all methods.

[1]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Masatoshi Okutomi,et al.  Significance and attributes of subpixel estimation on area-based matching , 2003, Systems and Computers in Japan.

[3]  Markus H. Gross,et al.  Online view sampling for estimating depth from light fields , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[4]  Chao Li,et al.  Robust depth estimation for light field via spinning parallelogram operator , 2016, Comput. Vis. Image Underst..

[5]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[6]  Bastian Goldlücke,et al.  Accurate Depth and Normal Maps from Occlusion-Aware Focal Stack Symmetry , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Sven Wanner,et al.  Globally consistent depth labeling of 4D light fields , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Can Chen,et al.  Depth Recovery from Light Field Using Focal Stack Symmetry , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[9]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[10]  Xiaoyan Hu,et al.  A Quantitative Evaluation of Confidence Measures for Stereo Vision , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Hiroshi Ishikawa,et al.  Exact Optimization for Markov Random Fields with Convex Priors , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Jitendra Malik,et al.  Depth from Combining Defocus and Correspondence Using Light-Field Cameras , 2013, 2013 IEEE International Conference on Computer Vision.

[13]  Touradj Ebrahimi,et al.  New Light Field Image Dataset , 2016, QoMEX 2016.

[14]  Ze-Nian Li,et al.  Continuous depth map reconstruction from light fields , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[15]  In-So Kweon,et al.  Accurate depth map estimation from a lenslet light field camera , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Leif Kobbelt,et al.  Image selection for improved Multi-View Stereo , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Moncef Gabbouj,et al.  Weighted median filters: a tutorial , 1996 .

[18]  In So Kweon,et al.  Depth from a Light Field Image with Learning-Based Matching Costs , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.

[20]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  In-So Kweon,et al.  EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[22]  Qi Zhang,et al.  100+ Times Faster Weighted Median Filter (WMF) , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Bastian Goldlücke,et al.  What Sparse Light Field Coding Reveals about Scene Structure , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Leonard McMillan,et al.  Dynamically reparameterized light fields , 2000, SIGGRAPH.

[25]  Katsuya Kondo,et al.  Depth map estimation using census transform for light field cameras , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[26]  Chao-Tsung Huang Robust Pseudo Random Fields for Light-Field Stereo Matching , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[27]  L. Armijo Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[28]  Alexei A. Efros,et al.  Occlusion-Aware Depth Estimation Using Light-Field Cameras , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[29]  Bastian Goldlücke,et al.  A Dataset and Evaluation Methodology for Depth Estimation on 4D Light Fields , 2016, ACCV.

[30]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.