Light Field Blind Deconvolution

We address for the first time the issue of motion blur in light field images captured from plenoptic cameras (instead of camera arrays), where the spatial sampling in each view is decimated. We propose a solution to the estimation of a sharp light field given a blurry one, when the motion blur point spread function is unknown, i.e., the so-called blind deconvolution problem. Unfortunately, the (decimated) spatial sampling of each light field view does not allow the use of current blind deconvolution approaches for traditional cameras. Also, due to the complexity of the imaging model, we investigate first the case of uniform (shift-invariant) blur of Lambertian objects, i.e., when objects are sufficiently far away from the camera to be approximately invariant to depth changes and their reflectance does not vary with the viewing direction. We introduce a highly parallelizable model for light field motion blur that is computationally and memory efficient. We then adapt a regularized blind deconvolution approach to our model and demonstrate its performance on both synthetic and real light field data.

[1]  Daniel Cremers,et al.  A Super-Resolution Framework for High-Accuracy Multiview Reconstruction , 2013, International Journal of Computer Vision.

[2]  Tae Hyun Kim,et al.  Dynamic Scene Deblurring , 2013, 2013 IEEE International Conference on Computer Vision.

[3]  Mostafa Kaveh,et al.  A regularization approach to joint blur identification and image restoration , 1996, IEEE Trans. Image Process..

[4]  Tom E. Bishop,et al.  The Light Field Camera: Extended Depth of Field, Aliasing, and Superresolution , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Eero P. Simoncelli,et al.  On Advances in Statistical Modeling of Natural Images , 2004, Journal of Mathematical Imaging and Vision.

[6]  William T. Freeman,et al.  Removing camera shake from a single photograph , 2006, SIGGRAPH 2006.

[7]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[8]  Stefan B. Williams,et al.  Decoding, Calibration and Rectification for Lenselet-Based Plenoptic Cameras , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Bernhard Schölkopf,et al.  Fast removal of non-uniform camera shake , 2011, 2011 International Conference on Computer Vision.

[10]  Tony F. Chan,et al.  Total variation blind deconvolution , 1998, IEEE Trans. Image Process..

[11]  Jean Ponce,et al.  Non-uniform Deblurring for Shaken Images , 2012, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Edward H. Adelson,et al.  Single Lens Stereo with a Plenoptic Camera , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Frédo Durand,et al.  Efficient marginal likelihood optimization in blind deconvolution , 2011, CVPR 2011.

[14]  Michael S. Brown,et al.  Richardson-Lucy Deblurring for Scenes under a Projective Motion Path , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Wei Xiong,et al.  Rotational Motion Deblurring of a Rigid Object from a Single Image , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[16]  Ankit Gupta,et al.  Single Image Deblurring Using Motion Density Functions , 2010, ECCV.

[17]  Thomas Pock,et al.  Variational Shape from Light Field , 2013, EMMCVPR.

[18]  Andrew Lumsdaine,et al.  Superresolution with the focused plenoptic camera , 2011, Electronic Imaging.

[19]  Mostafa Kaveh,et al.  Anisotropic blind image restoration , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[20]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[21]  Kang Wang,et al.  A two-stage approach to blind spatially-varying motion deblurring , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Aaron S. Andalman,et al.  Wave optics theory and 3-D deconvolution for the light field microscope. , 2013, Optics express.

[23]  Frédo Durand,et al.  Understanding Blind Deconvolution Algorithms , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  A. Lumsdaine Full Resolution Lightfield Rendering , 2008 .

[25]  Sunghyun Cho,et al.  Edge-based blur kernel estimation using patch priors , 2013, IEEE International Conference on Computational Photography (ICCP).

[26]  Shree K. Nayar,et al.  PiCam , 2013 .

[27]  Jiaya Jia,et al.  High-quality motion deblurring from a single image , 2008, ACM Trans. Graph..

[28]  Bernhard Schölkopf,et al.  Recording and Playback of Camera Shake: Benchmarking Blind Deconvolution with a Real-World Database , 2012, ECCV.

[29]  Ming-Hsuan Yang,et al.  Fast Non-uniform Deblurring using Constrained Camera Pose Subspace , 2012, BMVC.

[30]  Marc Levoy,et al.  Using plane + parallax for calibrating dense camera arrays , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[31]  Li Xu,et al.  Two-Phase Kernel Estimation for Robust Motion Deblurring , 2010, ECCV.

[32]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[33]  Sunghyun Cho,et al.  Fast motion deblurring , 2009, SIGGRAPH 2009.

[34]  Frédo Durand,et al.  Understanding and evaluating blind deconvolution algorithms , 2009, CVPR.

[35]  Bernhard Schölkopf,et al.  Efficient filter flow for space-variant multiframe blind deconvolution , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.