Adopting an unconstrained ray model in light-field cameras for 3D shape reconstruction

Given the raising interest in light-field technology and the increasing availability of professional devices, a feasible and accurate calibration method is paramount to unleash practical applications. In this paper we propose to embrace a fully non-parametric model for the imaging and we show that it can be properly calibrated with little effort using a dense active target. This process produces a dense set of independent rays that cannot be directly used to produce a conventional image. However, they are an ideal tool for 3D reconstruction tasks, since they are highly redundant, very accurate and they cover a wide range of different baselines. The feasibility and convenience of the process and the accuracy of the obtained calibration are comprehensively evaluated through several experiments.

[1]  Tom E. Bishop,et al.  Plenoptic depth estimation from multiple aliased views , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[2]  In So Kweon,et al.  Geometric Calibration of Micro-Lens-Based Light-Field Cameras Using Line Features , 2014, ECCV.

[3]  Tom E. Bishop,et al.  The Light Field Camera: Extended Depth of Field, Aliasing, and Superresolution , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Andrea Torsello,et al.  Can a Fully Unconstrained Imaging Model Be Applied Effectively to Central Cameras? , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[6]  Shree K. Nayar,et al.  A general imaging model and a method for finding its parameters , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[7]  Sven Wanner,et al.  Spatial and Angular Variational Super-Resolution of 4D Light Fields , 2012, ECCV.

[8]  Bastian Goldlücke,et al.  On the Calibration of Focused Plenoptic Cameras , 2013, Time-of-Flight and Depth Imaging.

[9]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[10]  Andrea Torsello,et al.  A Non-cooperative Game for 3D Object Recognition in Cluttered Scenes , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.

[11]  Olivier D. Faugeras,et al.  Automatic calibration and removal of distortion from scenes of structured environments , 1995, Optics & Photonics.

[12]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, SIGGRAPH 2007.

[13]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, SIGGRAPH 2005.

[14]  Yu-Wing Tai,et al.  Modeling the Calibration Pipeline of the Lytro Camera for High Quality Light-Field Image Reconstruction , 2013, 2013 IEEE International Conference on Computer Vision.

[15]  Bernd Michaelis,et al.  Optical 3D Surface Reconstruction by a Multi-Period Phase Shift Method , 2007, J. Comput..

[16]  Stefan B. Williams,et al.  Decoding, Calibration and Rectification for Lenselet-Based Plenoptic Cameras , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Sven Wanner,et al.  Globally consistent depth labeling of 4D light fields , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Gary R. Bradski,et al.  Learning OpenCV - computer vision with the OpenCV library: software that sees , 2008 .

[19]  O. Faugeras,et al.  Straight Lines Have to Be Straight Automatic Calibration and Removal of Distortion from Scenes of Structured Environments , 2001 .

[20]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[21]  E BishopTom,et al.  The Light Field Camera , 2012 .

[22]  Andrea Torsello,et al.  Sampling Relevant Points for Surface Registration , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.

[23]  Edward Jones,et al.  Accuracy of fish-eye lens models. , 2010, Applied optics.

[24]  Edmund Y. Lam,et al.  A high-resolution lightfield camera with dual-mask design , 2012, Other Conferences.

[25]  Marc Levoy,et al.  Using plane + parallax for calibrating dense camera arrays , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[26]  Chia-Kai Liang,et al.  Programmable aperture photography: multiplexed light field acquisition , 2008, SIGGRAPH 2008.

[27]  Ramesh Raskar,et al.  Axial-cones: modeling spherical catadioptric cameras for wide-angle light field rendering , 2010, SIGGRAPH 2010.

[28]  D. Shepard.  A two-dimensional interpolation function for irregularly-spaced data , 1968, ACM National Conference.

[29]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[30]  Zhan Yu,et al.  Light Field Stereo Matching Using Bilateral Statistics of Surface Cameras , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Frédo Durand,et al.  Linear view synthesis using a dimensionality gap light field prior , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[33]  Andrea Torsello,et al.  Multiview registration via graph diffusion of dual quaternions , 2011, CVPR 2011.

[34]  Peter F. Sturm,et al.  A Generic Concept for Camera Calibration , 2004, ECCV.