Multi-view Image Restoration from Plenoptic Raw Images

We present a reconstruction algorithm that can restore the captured 4D light field from a portable plenoptic camera without the need for calibration images. An efficient and robust estimator is proposed to accurately detect the centers of microlens images. Based on that estimator, parameters that model the centers of microlens array images are obtained by solving a global optimization problem. To further enhance the quality of reconstructed multi-view images, a novel 4D demosaicing algorithm based on kernel regression is also proposed. Our experimental results show that it outperforms state of the art algorithms.

[1]  Zhan Yu,et al.  An analysis of color demosaicing in plenoptic cameras , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  In So Kweon,et al.  Geometric Calibration of Micro-Lens-Based Light-Field Cameras Using Line Features , 2014, ECCV.

[3]  G. Lippmann,et al.  Epreuves reversibles. Photographies integrals , 1908 .

[4]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[5]  Chia-Kai Liang,et al.  Programmable aperture photography: multiplexed light field acquisition , 2008, SIGGRAPH 2008.

[6]  Jitendra Malik,et al.  Depth from Combining Defocus and Correspondence Using Light-Field Cameras , 2013, 2013 IEEE International Conference on Computer Vision.

[7]  C. S. Garbe,et al.  Plenoptic Particle Streak Velocimetry ( pPSV ) : 3 D 3 C fluid flow measurement from light fields with a single plenoptic camera , 2012 .

[8]  Yu-Wing Tai,et al.  Modeling the Calibration Pipeline of the Lytro Camera for High Quality Light-Field Image Reconstruction , 2013, 2013 IEEE International Conference on Computer Vision.

[9]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[10]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[11]  Christian B Allen,et al.  48th AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition , 2010 .

[12]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[13]  Zhan Yu,et al.  Lytro camera technology: theory, algorithms, performance analysis , 2013, Electronic Imaging.

[14]  Zhan Yu,et al.  Line Assisted Light Field Triangulation and Stereo Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[15]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, SIGGRAPH 2007.

[16]  Brian S. Thurow,et al.  Three-Dimensional Particle Image Velocimetry Using a Plenoptic Camera , 2012 .

[17]  Patrick Pérez,et al.  Light-Field Demultiplexing and Disparity Estimation , 2014 .

[18]  A. Lumsdaine.  Full Resolution Lightfield Rendering , 2008 .

[19]  Stefan B. Williams,et al.  Decoding, Calibration and Rectification for Lenselet-Based Plenoptic Cameras , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Xiang Huang,et al.  Dictionary Learning Based Color Demosaicing for Plenoptic Cameras , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[21]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[22]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[23]  Richard Szeliski,et al.  Refocusing plenoptic images using depth-adaptive splatting , 2014, 2014 IEEE International Conference on Computational Photography (ICCP).