Advanced light-field refocusing through tomographic modeling of the photographed scene

Recently we have shown that light-field photography images can be interpreted as limited-angle cone-beam tomography acquisitions. Here, we use this property to develop a direct-space tomographic refocusing formulation that allows one to refocus both unfocused and focused light-field images. We express the reconstruction as a convex optimization problem, thus enabling the use of various regularization terms to help suppress artifacts, and a wide class of existing advanced tomographic algorithms. This formulation also supports super-resolved reconstructions and the correction of the optical system’s limited frequency response (point spread function). We validate this method with numerical and real-world examples. © 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

[1]  J. Goodman.  Introduction to Fourier optics , 1969 .

[2]  H. V. D. Vorst,et al.  SIRT- and CG-type methods for the iterative solution of sparse linear least-squares problems , 1990 .

[3]  C. Burrus,et al.  Noise reduction using an undecimated discrete wavelet transform , 1996, IEEE Signal Processing Letters.

[4]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[5]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[6]  Ren Ng.  Fourier Slice Photography , 2005 .

[7]  P. Hanrahan,et al.  Digital light field photography , 2006 .

[8]  Edmund Y. Lam,et al.  Super-resolution reconstruction in a computational compound-eye imaging system , 2007, Multidimens. Syst. Signal Process..

[9]  Tom E. Bishop,et al.  Light field superresolution , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[10]  Andrew Lumsdaine,et al.  The focused plenoptic camera , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[11]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[12]  Andrew Lumsdaine,et al.  Focused plenoptic camera and rendering , 2010, J. Electronic Imaging.

[13]  K J Batenburg,et al.  Performance improvements for iterative electron tomography reconstruction using graphics processing units (GPUs). , 2011, Journal of structural biology.

[14]  Andrew Lumsdaine,et al.  Superresolution with the focused plenoptic camera , 2011, Electronic Imaging.

[15]  Andrew Lumsdaine,et al.  Spatial analysis of discrete plenoptic sampling , 2011, Electronic Imaging.

[16]  Sven Wanner,et al.  Spatial and Angular Variational Super-Resolution of 4D Light Fields , 2012, ECCV.

[17]  E. Sidky,et al.  Convex optimization problem prototyping for image reconstruction in computed tomography with the Chambolle–Pock algorithm , 2011, Physics in medicine and biology.

[18]  Gordon Wetzstein,et al.  Compressive light field photography using overcomplete dictionaries and optimized projections , 2013, ACM Trans. Graph..

[19]  Aaron S. Andalman,et al.  Wave optics theory and 3-D deconvolution for the light field microscope. , 2013, Optics express.

[20]  Stefan B. Williams,et al.  Decoding, Calibration and Rectification for Lenselet-Based Plenoptic Cameras , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Jitendra Malik,et al.  Depth from Combining Defocus and Correspondence Using Light-Field Cameras , 2013, 2013 IEEE International Conference on Computer Vision.

[22]  Kathrin Berkner,et al.  Image formation analysis and high resolution image reconstruction for plenoptic imaging systems. , 2013, Applied optics.

[23]  Edmund Y Lam,et al.  Computational photography with plenoptic camera and light field capture: tutorial. , 2015, Journal of the Optical Society of America. A, Optics, image science, and vision.

[24]  Stefan B. Williams,et al.  Linear Volumetric Focus for Light Field Cameras , 2015, TOGS.

[25]  Tieniu Tan,et al.  A simple and robust super resolution method for light field images , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[26]  In-So Kweon,et al.  Light-Field Image Super-Resolution Using Convolutional Neural Network , 2017, IEEE Signal Processing Letters.

[27]  Pascal Frossard,et al.  Geometry-Consistent Light Field Super-Resolution via Graph-Based Regularization , 2017, IEEE Transactions on Image Processing.

[28]  Kees Joost Batenburg,et al.  Tomographic approach for the quantitative scene reconstruction from light field images. , 2018, Optics express.

[29]  Liang Gao,et al.  On the fundamental comparison between unfocused and focused light field cameras. , 2018, Applied optics.

[30]  Xavier Levecq,et al.  Comparison of reconstruction approaches for plenoptic imaging systems , 2018, Photonics Europe.

[31]  Bahram Javidi,et al.  Fundamentals of 3D imaging and displays: a tutorial on integral imaging, light-field, and plenoptic systems , 2018 .

[32]  P. Alam.  ‘O’ , 2021, Composites Engineering: An A–Z Guide.