Homography-based model with light calibration for plenoptic cameras

Abstract. Light-field and plenoptic cameras are widely available today. Compared with monocular cameras, these cameras capture not only the intensity but also the direction of the light rays. Due to this specificity, light-field cameras allow for image refocusing and depth estimation using a single image. However, most of the existing depth estimation methods using light-field cameras require a prior complex calibration phase and raw data preprocessing before the desired algorithm is applied. We propose a homography-based method with plenoptic camera parameters calibration and optimization, dedicated to our homography-based micro-images matching algorithm. The proposed method works on debayerred raw images with vignetting correction. The proposed approach directly links the disparity estimation in the 2D image plane to the depth estimation in the 3D object plane, allowing for direct extraction of the real depth without any intermediate virtual depth estimation phase. Also, calibration parameters used in the depth estimation algorithm are directly estimated, and hence no prior complex calibration is needed. Results are illustrated by performing depth estimation with a focused light-field camera over a large distance range up to 4 m.

[1]  Charles T. Loop,et al.  Computing rectifying homographies for stereo vision , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[2]  Yo-Sung Ho,et al.  Efficient Depth Estimation for Light Field Images , 2018, 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC).

[3]  A. Lumsdaine.  Full Resolution Lightfield Rendering , 2008 .

[4]  P. Hanrahan,et al.  Digital light field photography , 2006 .

[5]  Edward H. Adelson,et al.  Single Lens Stereo with a Plenoptic Camera , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Bastian Goldlücke,et al.  On the Calibration of Focused Plenoptic Cameras , 2013, Time-of-Flight and Depth Imaging.

[7]  Omar Ait-Aider,et al.  Blur Aware Calibration of Multi-Focus Plenoptic Camera , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Andrew Lumsdaine,et al.  The focused plenoptic camera , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[9]  Sven Wanner,et al.  Variational Light Field Analysis for Disparity Estimation and Super-Resolution , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  In-So Kweon,et al.  Geometric Calibration of Micro-Lens-Based Light Field Cameras Using Line Features , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[12]  A. Dieterlen,et al.  Interest of pseudo-focused images for key-points detection in plenoptic imaging , 2021, International Conference on Quality Control by Artificial Vision.

[13]  Vladan Velisavljevic,et al.  Real-Time Refocusing Using an FPGA-Based Standard Plenoptic Camera , 2018, IEEE Transactions on Industrial Electronics.

[14]  Thomas Pock,et al.  Shape from Light Field Meets Robust PCA , 2014, ECCV.

[15]  Zezhi Chen,et al.  Uncalibrated two-view metrology , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[16]  Peng Sun,et al.  Light-field-depth-estimation network based on epipolar geometry and image segmentation. , 2020, Journal of the Optical Society of America. A, Optics, image science, and vision.

[17]  Sébastien Ourselin,et al.  Corner-Based Geometric Calibration of Multi-focus Plenoptic Cameras , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[18]  Stefan B. Williams,et al.  Decoding, Calibration and Rectification for Lenselet-Based Plenoptic Cameras , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Uwe Stilla,et al.  Establishing a Probabilistic Depth Map from Focused Plenoptic Cameras , 2015, 2015 International Conference on 3D Vision.

[20]  Uwe Stilla,et al.  METRIC CALIBRATION OF A FOCUSED PLENOPTIC CAMERA BASED ON A 3D CALIBRATION TARGET , 2016 .

[21]  Shengxian Shi,et al.  Parametric study on light field volumetric particle image velocimetry , 2016 .

[22]  Jonathan Nissanov,et al.  Symmetry-based 3D brain reconstruction , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[23]  Ivana Tosic,et al.  Light Field Scale-Depth Space Transform for Dense Depth Estimation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[24]  Uwe Stilla,et al.  Calibration and accuracy analysis of a focused plenoptic camera , 2014 .

[25]  In-So Kweon,et al.  Accurate depth map estimation from a lenslet light field camera , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Sven Wanner,et al.  Globally consistent depth labeling of 4D light fields , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Michel Dhome,et al.  Light-Field Camera Calibration from Raw Images , 2017, 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[28]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..