A Generic Multi-Projection-Center Model and Calibration Method for Light Field Cameras

Light field cameras can capture both spatial and angular information of light rays, enabling 3D reconstruction by a single exposure. The geometry of 3D reconstruction is affected by intrinsic parameters of a light field camera significantly. In the paper, we propose a multi-projection-center (MPC) model with 6 intrinsic parameters to characterize light field cameras based on traditional two-parallel-plane (TPP) representation. The MPC model can generally parameterize light field in different imaging formations, including conventional and focused light field cameras. By the constraints of 4D ray and 3D geometry, a 3D projective transformation is deduced to describe the relationship between geometric structure and the MPC coordinates. Based on the MPC model and projective transformation, we propose a calibration algorithm to verify our light field camera model. Our calibration method includes a close-form solution and a non-linear optimization by minimizing re-projection errors. Experimental results on both simulated and real scene data have verified the performance of our algorithm.

[1]  Zhan Yu,et al.  Ieee Transactions on Visualization and Computer Graphics 1 Enhancing Light Fields through Ray-space Stitching , 2022 .

[2]  Ren Ng.  Fourier slice photography , 2005, ACM Trans. Graph..

[3]  Andrew Lumsdaine,et al.  The focused plenoptic camera , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[4]  Richard I. Hartley,et al.  Self-Calibration of Stationary Cameras , 1997, International Journal of Computer Vision.

[5]  Vladan Velisavljevic,et al.  The refocusing distance of a standard plenoptic photograph , 2015, 2015 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[6]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[7]  Tom E. Bishop,et al.  The Light Field Camera: Extended Depth of Field, Aliasing, and Superresolution , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Marc Levoy,et al.  Using plane + parallax for calibrating dense camera arrays , 2004, CVPR 2004.

[9]  Sven Wanner,et al.  Globally Consistent Multi-label Assignment on the Ray Space of 4D Light Fields , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[11]  Stephan Hussmann,et al.  Automated robust metric calibration of multi-focus plenoptic cameras , 2015, 2015 IEEE International Instrumentation and Measurement Technology Conference (I2MTC) Proceedings.

[12]  Bernd Jähne,et al.  Generating EPI Representations of 4D Light Fields with a Single Lens Focused Plenoptic Camera , 2011, ISVC.

[13]  Clemens Birklbauer,et al.  Panorama light-field imaging , 2012, SIGGRAPH '12.

[14]  In-So Kweon,et al.  Accurate depth map estimation from a lenslet light field camera , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Brian S. Thurow,et al.  Calibration of a Microlens Array for a Plenoptic Camera , 2014 .

[16]  In-So Kweon,et al.  Geometric Calibration of Micro-Lens-Based Light Field Cameras Using Line Features , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Robert Pless,et al.  Using many cameras as one , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[18]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, ACM Trans. Graph..

[19]  Yu-Wing Tai,et al.  Modeling the Calibration Pipeline of the Lytro Camera for High Quality Light-Field Image Reconstruction , 2013, 2013 IEEE International Conference on Computer Vision.

[20]  Qing Wang,et al.  Decoding and calibration method on focused plenoptic camera , 2016, Computational Visual Media.

[21]  Stefan B. Williams,et al.  Decoding, Calibration and Rectification for Lenselet-Based Plenoptic Cameras , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Kaj Madsen,et al.  Methods for Non-Linear Least Squares Problems , 1999 .

[23]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[24]  S. P. Mudur,et al.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[25]  Bastian Goldlücke,et al.  On Linear Structure from Motion for Light Field Cameras , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[26]  Bastian Goldlücke,et al.  On the Calibration of Focused Plenoptic Cameras , 2013, Time-of-Flight and Depth Imaging.

[27]  P. Hanrahan,et al.  Digital light field photography , 2006 .

[28]  Vladan Velisavljevic,et al.  Light field geometry of a Standard Plenoptic Camera. , 2014, Optics express.

[29]  Marc Levoy,et al.  Using plane + parallax for calibrating dense camera arrays , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..