Deblurring Low-Light Images with Light Streaks

Images acquired in low-light conditions with handheld cameras are often blurry, so steady poses and long exposure time are required to alleviate this problem. Although significant advances have been made in image deblurring, state-of-the-art approaches often fail on low-light images, as a sufficient number of salient features cannot be extracted for blur kernel estimation. On the other hand, light streaks are common phenomena in low-light images that have not been extensively explored in existing approaches. In this work, we propose an algorithm that utilizes light streaks to facilitate deblurring low-light images. The light streaks, which commonly exist in the low-light blurry images, contain rich information regarding camera motion and blur kernels. A method is developed in this work to detect light streaks for kernel estimation. We introduce a non-linear blur model that explicitly takes light streaks and corresponding light sources into account, and pose them as constraints for estimating the blur kernel in an optimization framework. For practical applications, the proposed algorithm is extended to handle images undergoing non-uniform blur. Experimental results show that the proposed algorithm performs favorably against the state-of-the-art methods on deblurring real-world low-light images.

[1]  Bernhard Schölkopf,et al.  Fast removal of non-uniform camera shake , 2011, 2011 International Conference on Computer Vision.

[2]  Seungyong Lee,et al.  Handling outliers in non-blind image deconvolution , 2011, 2011 International Conference on Computer Vision.

[3]  Bernhard Schölkopf,et al.  Space-Variant Single-Image Blind Deconvolution for Removing Camera Shake , 2010, NIPS.

[4]  Sylvain Paris,et al.  Blur kernel estimation using the radon transform , 2011, CVPR 2011.

[5]  Michael S. Brown,et al.  Richardson-Lucy Deblurring for Scenes under a Projective Motion Path , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Richard Szeliski,et al.  PSF estimation using sharp edge prediction , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Bernhard Schölkopf,et al.  Multiframe blind deconvolution, super-resolution, and saturation correction via incremental EM , 2010, 2010 IEEE International Conference on Image Processing.

[8]  Yanning Zhang,et al.  Multi-Observation Blind Deconvolution with an Adaptive Sparse Prior , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Frédo Durand,et al.  Understanding and evaluating blind deconvolution algorithms , 2009, CVPR.

[10]  Frédo Durand,et al.  Efficient marginal likelihood optimization in blind deconvolution , 2011, CVPR 2011.

[11]  Yasuyuki Matsushita,et al.  Removing Non-Uniform Motion Blur from Images , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[12]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[13]  Ming-Hsuan Yang,et al.  Good Regions to Deblur , 2012, ECCV.

[14]  Ankit Gupta,et al.  Single Image Deblurring Using Motion Density Functions , 2010, ECCV.

[15]  Jian-Feng Cai,et al.  Blind motion deblurring from a single image using sparse approximation , 2009, CVPR.

[16]  Nanning Zheng,et al.  PSF Estimation via Gradient Domain Correlation , 2012, IEEE Transactions on Image Processing.

[17]  Li Xu,et al.  Unnatural L0 Sparse Representation for Natural Image Deblurring , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Jian-Feng Cai,et al.  Framelet-Based Blind Motion Deblurring From a Single Image , 2012, IEEE Transactions on Image Processing.

[19]  Ming-Hsuan Yang,et al.  Fast Non-uniform Deblurring using Constrained Camera Pose Subspace , 2012, BMVC.

[20]  Andrew Zisserman,et al.  Deblurring shaken and partially saturated images , 2011, ICCV Workshops.

[21]  Sunghyun Cho,et al.  Fast motion deblurring , 2009, SIGGRAPH 2009.

[22]  William T. Freeman,et al.  Removing camera shake from a single photograph , 2006, SIGGRAPH 2006.

[23]  Ramesh Raskar,et al.  Coded exposure photography: motion deblurring using fluttered shutter , 2006, SIGGRAPH 2006.

[24]  G. J. Burton,et al.  Color and spatial structure in natural scenes. , 1987, Applied optics.

[25]  Rob Fergus,et al.  Blind deconvolution using a normalized sparsity measure , 2011, CVPR 2011.

[26]  Li Xu,et al.  Two-Phase Kernel Estimation for Robust Motion Deblurring , 2010, ECCV.

[27]  Michael S. Brown,et al.  Nonlinear Camera Response Functions and Image Deblurring: Theoretical Analysis and Practice , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Hui Ma,et al.  Image Deblurring with Blurred / Noisy Image Pairs , 2013 .

[29]  Bernhard Schölkopf,et al.  Recording and Playback of Camera Shake: Benchmarking Blind Deconvolution with a Real-World Database , 2012, ECCV.

[30]  Jean Ponce,et al.  Non-uniform Deblurring for Shaken Images , 2012, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Jiaya Jia,et al.  High-quality motion deblurring from a single image , 2008, ACM Trans. Graph..

[32]  Wei Xiong,et al.  Rotational Motion Deblurring of a Rigid Object from a Single Image , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[33]  Richard Szeliski,et al.  A content-aware image prior , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Stephen Lin,et al.  Image/video deblurring using a hybrid camera , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Kok-Lim Low,et al.  Interactive motion deblurring using light streaks , 2011, 2011 18th IEEE International Conference on Image Processing.

[36]  Stuart Jefferies,et al.  A computational method for the restoration of images with an unknown, spatially-varying blur. , 2006, Optics express.

[37]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, SIGGRAPH 2007.

[38]  S. B. Kang,et al.  Image deblurring using inertial measurement sensors , 2010, SIGGRAPH 2010.

[39]  Anat Levin,et al.  Blind Motion Deblurring Using Image Statistics , 2006, NIPS.

[40]  N. Kopeika,et al.  Direct method for restoration of motion-blurred images , 1998 .

[41]  Raanan Fattal,et al.  Blur-Kernel Estimation from Spectral Irregularities , 2012, ECCV.