Robust blind motion deblurring using near-infrared flash image

In light-limited situations, camera motion blur is one of the prime causes for poor image quality. Recovering the blur kernel and latent image from the blurred observation is an inherently ill-posed problem. In this paper, we introduce a hand-held multispectral camera to capture a pair of blurred image and Near-InfraRed (NIR) flash image simultaneously and analyze the correlation between the pair of images. To utilize the high-frequency details of the scene captured by the NIR-flash image, we exploit the NIR gradient constraint as a new type of image regularization, and integrate it into a Maximum-A-Posteriori (MAP) problem to iteratively perform the kernel estimation and image restoration. We demonstrate our method on the synthetic and real images with both spatially invariant and spatially varying blur. The experiments strongly support the effectiveness of our method to provide both accurate kernel estimation and superior latent image with more details and fewer ringing artifacts.

[1]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, SIGGRAPH 2007.

[2]  Michael F. Cohen,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[3]  Rayan Saab,et al.  Stable sparse approximations via nonconvex optimization , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[4]  Dani Lischinski,et al.  The Shadow Meets the Mask: Pyramid‐Based Shadow Removal , 2008, Comput. Graph. Forum.

[5]  M. Simon.  Probability distributions involving Gaussian random variables : a handbook for engineers and scientists , 2002 .

[6]  S. B. Kang,et al.  Image deblurring using inertial measurement sensors , 2010, SIGGRAPH 2010.

[7]  Jian-Feng Cai,et al.  Blind motion deblurring from a single image using sparse approximation , 2009, CVPR.

[8]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[9]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.

[10]  Jia Chen,et al.  Robust dual motion deblurring , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  William H. Richardson,et al.  Bayesian-Based Iterative Method of Image Restoration , 1972 .

[12]  Jian Sun,et al.  Progressive inter-scale and intra-scale non-blind image deconvolution , 2008, SIGGRAPH 2008.

[13]  Bernhard Schölkopf,et al.  Space-Variant Single-Image Blind Deconvolution for Removing Camera Shake , 2010, NIPS.

[14]  Sabine Süsstrunk,et al.  Illuminant estimation and detection using near-infrared , 2009, Electronic Imaging.

[15]  R. Fergus,et al.  Dark flash photography , 2009, ACM Trans. Graph..

[16]  Jiaya Jia,et al.  High-quality motion deblurring from a single image , 2008, ACM Trans. Graph..

[17]  Sunghyun Cho,et al.  Fast motion deblurring , 2009, SIGGRAPH 2009.

[18]  Sabine Süsstrunk,et al.  Automatic skin enhancement with visible and near-infrared image fusion , 2010, ACM Multimedia.

[19]  Shree K. Nayar,et al.  Motion-based motion deblurring , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Jean Ponce,et al.  Non-uniform Deblurring for Shaken Images , 2012, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Sabine Süsstrunk,et al.  Multi-spectral SIFT for scene category recognition , 2011, CVPR 2011.

[22]  Shmuel Peleg,et al.  Two motion-blurred images are better than one , 2005, Pattern Recognit. Lett..

[23]  Richard Szeliski,et al.  PSF estimation using sharp edge prediction , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Sabine Süsstrunk,et al.  Colouring the Near-Infrared , 2008, CIC.

[25]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series , 1964 .

[26]  Jian-Feng Cai,et al.  High-quality curvelet-based motion deblurring from an image pair , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Sabine Süsstrunk,et al.  Color image dehazing using the near-infrared , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[28]  Frédo Durand,et al.  Understanding and evaluating blind deconvolution algorithms , 2009, CVPR.

[29]  Yo-Sung Ho,et al.  High-quality non-blind image deconvolution with adaptive regularization , 2011, J. Vis. Commun. Image Represent..

[30]  Stephen Lin,et al.  Correction of Spatially Varying Image and Video Motion Blur Using a Hybrid Camera , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Michael S. Brown,et al.  Richardson-Lucy Deblurring for Scenes under a Projective Motion Path , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Yehoshua Y. Zeevi,et al.  Image enhancement and denoising by complex diffusion processes , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Takahiro Okabe,et al.  Image Enhancement of Low-light Scenes with Near-infrared Flash Images , 2009, IPSJ Trans. Comput. Vis. Appl..

[34]  Ankit Gupta,et al.  Single Image Deblurring Using Motion Density Functions , 2010, ECCV.

[35]  Hui Ma,et al.  Image Deblurring with Blurred / Noisy Image Pairs , 2013 .

[36]  Sabine Süsstrunk,et al.  Material Classification Using Color and NIR Images , 2009, CIC.

[37]  Sabine Süsstrunk,et al.  Automatic and Accurate Shadow Detection from (potentially) a Single Image Using Near-infrared Information , 2022 .

[38]  Jiaya Jia,et al.  Single Image Motion Deblurring Using Transparency , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Bernhard Schölkopf,et al.  Fast removal of non-uniform camera shake , 2011, 2011 International Conference on Computer Vision.

[40]  James H. Money,et al.  Total variation minimizing blind deconvolution with shock filter reference , 2008, Image Vis. Comput..

[41]  Dong Guo,et al.  Robust flash deblurring , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[42]  William T. Freeman,et al.  Removing camera shake from a single photograph , 2006, SIGGRAPH 2006.

[43]  Rob Fergus,et al.  Blind deconvolution using a normalized sparsity measure , 2011, CVPR 2011.

[44]  Li Xu,et al.  Two-Phase Kernel Estimation for Robust Motion Deblurring , 2010, ECCV.

[45]  Richard Szeliski,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[46]  Harry Shum,et al.  Blurred/Non-Blurred Image Alignment using Sparseness Prior , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[47]  Xiaopeng Zhang,et al.  Enhancing photographs with Near Infra-Red images , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Sabine Süsstrunk,et al.  8.1: Invited Paper: Enhancing the Visible with the Invisible: Exploiting Near Infrared to Advance Computational Photography and Computer Vision , 2010 .

[49]  Weiguo Gong,et al.  Total variation blind deconvolution employing split Bregman iteration , 2012, J. Vis. Commun. Image Represent..

[50]  Junfeng Yang,et al.  An Efficient TVL1 Algorithm for Deblurring Multichannel Images Corrupted by Impulsive Noise , 2009, SIAM J. Sci. Comput..

[51]  Xiaopeng Zhang,et al.  Enhancing low light images using near infrared flash images , 2010, 2010 IEEE International Conference on Image Processing.