Rapid Light Field Depth Estimation with Semi-Global Matching

Running time of the light field depth estimation algorithms is typically high. This assessment is based on the computational complexity of existing methods and the large amounts of data involved. The aim of our work is to develop a simple and fast algorithm for accurate depth computation. In this context, we propose an approach, which involves Semi-Global Matching for the processing of light field images. It forms on comparison of pixels’ correspondences with different metrics in the substantially bounded light field space. We show that our method is suitable for the fast production of a proper result in a variety of light field configurations.

[1]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[2]  H. Hirschmüller.  Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information , 2005, CVPR.

[3]  D. Yang,et al.  Occlusion-aware depth estimation for light field using multi-orientation EPIs , 2018, Pattern Recognit..

[4]  Heiko Hirschmüller,et al.  Evaluation of Cost Functions for Stereo Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Katsuya Kondo,et al.  Depth map estimation using census transform for light field cameras , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Antonio M. López,et al.  Embedded Real-time Stereo Estimation via Semi-Global Matching on the GPU , 2016, ICCS.

[7]  Chao-Tsung Huang.  Robust Pseudo Random Fields for Light-Field Stereo Matching , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  M. Levoy,et al.  The light field , 1939 .

[9]  Yael Pritch,et al.  Scene reconstruction from high spatio-angular resolution light fields , 2013, ACM Trans. Graph..

[10]  Bastian Goldlücke,et al.  Accurate Depth and Normal Maps from Occlusion-Aware Focal Stack Symmetry , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Mark Horowitz,et al.  High performance imaging using arrays of inexpensive cameras , 2004 .

[12]  Bastian Goldlücke,et al.  What Sparse Light Field Coding Reveals about Scene Structure , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Christopher Joseph Pal,et al.  Learning Conditional Random Fields for Stereo , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  In-So Kweon,et al.  EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Andrew Lumsdaine,et al.  The focused plenoptic camera , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[16]  Edmund Y. Lam,et al.  Data-driven light field depth estimation using deep Convolutional Neural Networks , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[17]  F. Bethmann,et al.  Semi-Global Matching in Object Space , 2015 .

[18]  In So Kweon,et al.  Depth from a Light Field Image with Learning-Based Matching Costs , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[20]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.

[21]  Heiko Hirschmüller,et al.  Evaluation of Stereo Matching Costs on Images with Radiometric Differences , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Pushmeet Kohli,et al.  Markov Random Fields for Vision and Image Processing , 2011 .

[23]  In-So Kweon,et al.  Accurate depth map estimation from a lenslet light field camera , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Alexei A. Efros,et al.  Occlusion-Aware Depth Estimation Using Light-Field Cameras , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[25]  Bastian Goldlücke,et al.  A Dataset and Evaluation Methodology for Depth Estimation on 4D Light Fields , 2016, ACCV.

[26]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[27]  Thomas Pock,et al.  Convolutional Networks for Shape from Light Field , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Sven Wanner,et al.  Globally consistent depth labeling of 4D light fields , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Lennart Wietzke,et al.  Single lens 3D-camera with extended depth-of-field , 2012, Electronic Imaging.

[30]  Didier Stricker,et al.  Fast and Efficient Depth Map Estimation from Light Fields , 2017, 2017 International Conference on 3D Vision (3DV).

[31]  Didier Stricker,et al.  A Compact Light Field Camera for Real-Time Depth Estimation , 2019, CAIP.

[32]  Chaur-Chin Chen,et al.  Similarity Measurement Between Images , 2005, COMPSAC.

[33]  Alessandro Neri,et al.  A multi-resolution approach to depth field estimation in dense image arrays , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[34]  Neus Sabater,et al.  Dataset and Pipeline for Multi-view Light-Field Video , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[35]  Vladimir Kolmogorov,et al.  Multi-camera Scene Reconstruction via Graph Cuts , 2002, ECCV.

[36]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Touradj Ebrahimi,et al.  New Light Field Image Dataset , 2016, QoMEX 2016.

[38]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[39]  Shree K. Nayar,et al.  PiCam , 2013, ACM Trans. Graph..

[40]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[41]  V. A. Epanechnikov.  Non-Parametric Estimation of a Multivariate Probability Density , 1969 .

[42]  Sergiu Nedevschi,et al.  Real-time semi-global dense stereo solution with improved sub-pixel accuracy , 2010, 2010 IEEE Intelligent Vehicles Symposium.