Coded exposure deblurring: Optimized codes for PSF estimation and invertibility

We consider the problem of single image object motion deblurring from a static camera. It is well-known that deblurring of moving objects using a traditional camera is ill-posed, due to the loss of high spatial frequencies in the captured blurred image. A coded exposure camera modulates the integration pattern of light by opening and closing the shutter within the exposure time using a binary code. The code is chosen to make the resulting point spread function (PSF) invertible, for best deconvolution performance. However, for a successful deconvolution algorithm, PSF estimation is as important as PSF invertibility. We show that PSF estimation is easier if the resulting motion blur is smooth and the optimal code for PSF invertibility could worsen PSF estimation, since it leads to non-smooth blur. We show that both criterions of PSF invertibility and PSF estimation can be simultaneously met, albeit with a slight increase in the deconvolution noise. We propose design rules for a code to have good PSF estimation capability and outline two search criteria for finding the optimal code for a given length. We present theoretical analysis comparing the performance of the proposed code with the code optimized solely for PSF invertibility. We also show how to easily implement coded exposure on a consumer grade machine vision camera with no additional hardware. Real experimental results demonstrate the effectiveness of the proposed codes for motion deblurring.

[1]  William H. Richardson,et al.  Bayesian-Based Iterative Method of Image Restoration , 1972 .

[2]  L. Lucy An iterative technique for the rectification of observed distributions , 1974 .

[3]  T. M. Cannon,et al.  Coded aperture imaging with uniformly redundant arrays. , 1978, Applied optics.

[4]  H. Vincent Poor,et al.  An Introduction to Signal Detection and Estimation , 1994, Springer Texts in Electrical Engineering.

[5]  W. Cathey,et al.  Extended depth of field through wave-front coding. , 1995, Applied optics.

[6]  James F. Blinn,et al.  Blue screen matting , 1996, SIGGRAPH.

[7]  P. Jansson Deconvolution of images and spectra , 1997 .

[8]  R. Accorsi,et al.  Optimal coded aperture patterns for improved SNR in nuclear medicine imaging , 2001 .

[9]  Shree K. Nayar,et al.  A theory of multiplexed illumination , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[10]  Shree K. Nayar,et al.  Programmable imaging using a digital micromirror array , 2004, CVPR 2004.

[11]  Ramesh Raskar,et al.  Coded exposure photography: motion deblurring using fluttered shutter , 2006, SIGGRAPH 2006.

[12]  Shree K. Nayar,et al.  Lensless Imaging with a Controllable Aperture , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[13]  William T. Freeman,et al.  Removing camera shake from a single photograph , 2006, SIGGRAPH 2006.

[14]  Yoav Y. Schechner,et al.  Illumination Multiplexing within Fundamental Limits , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, SIGGRAPH 2007.

[16]  Jiaya Jia,et al.  Single Image Motion Deblurring Using Transparency , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, SIGGRAPH 2007.

[18]  Jian Sun,et al.  Progressive inter-scale and intra-scale non-blind image deconvolution , 2008, SIGGRAPH 2008.

[19]  Jiaya Jia,et al.  High-quality motion deblurring from a single image , 2008, ACM Trans. Graph..

[20]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2008 .

[21]  Richard Szeliski,et al.  PSF estimation using sharp edge prediction , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Chia-Kai Liang,et al.  Programmable aperture photography: multiplexed light field acquisition , 2008, SIGGRAPH 2008.

[23]  Ying Wu,et al.  Motion from blur , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Frédo Durand,et al.  Motion-invariant photography , 2008, ACM Trans. Graph..

[25]  Shree K. Nayar,et al.  Transactions on Pattern Analysis and Machine Intelligence Flexible Depth of Field Photography 1 Depth of Field , 2022 .