Blur kernel estimation of noisy-blurred image via dynamic structure prior

Abstract An accurate blur kernel is key to blind image deblurring and kernel estimation heavily relies on strong edges in the observed image [ 1 , 2, 3]. Previous methods [4] [5] leveraging image gradient prior with i.i.d statistics can hardly restrict strong edges in a noisy-blurred image, since both noise and strong edges are presented as strong gradients. In this paper, we propose a dynamic structure prior (DSP) to distinguish strong edges from noise by a non local total variation regularization. Our method measures accumulated intensity change of each pixel along all possible directions within its neighborhood. In this case, the structural contents with less change in certain directions, i.e.strong edges, will be accumulated lower than non-structural contents such as noise. By contrast, the previous image gradient priors only measure the changes in horizontal and vertical directions, which is not capable to present the directional characteristic of image edges. Moreover, in our method, the prior changes the weight dynamically with regard to local structures to impose fewer penalties on pixels with large changes. As a result, strong edges are preserved while fine textures are suppressed by large penalties. At last, a cost function containing the dynamic structure prior and L0 gradient prior that sharpens the preserved edges is proposed to estimate blur kernels from noisy-blurred images. Comprehensive experimental results show that our method outperforms previous methods in both commonly used datasets with various noise levels and real world images, from monochrome image to color image.

[1]  Marios Savvides,et al.  Douglas-Rachford Networks: Learning Both the Image Prior and Data Fidelity Terms for Blind Image Deconvolution , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Dani Lischinski,et al.  Deblurring by Example Using Dense Correspondence , 2013, 2013 IEEE International Conference on Computer Vision.

[3]  Yi Wang,et al.  Scale-Recurrent Network for Deep Image Deblurring , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[4]  Rob Fergus,et al.  Blind deconvolution using a normalized sparsity measure , 2011, CVPR 2011.

[5]  William T. Freeman,et al.  Removing camera shake from a single photograph , 2006, SIGGRAPH 2006.

[6]  Jian-Jiun Ding,et al.  Blur kernel estimation using normalized color-line priors , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Li Xu,et al.  Two-Phase Kernel Estimation for Robust Motion Deblurring , 2010, ECCV.

[8]  Vijayan Ellappan,et al.  Reconstruction of noisy and blurred images using blur kernel , 2017 .

[9]  Jean Ponce,et al.  Robust Guided Image Filtering Using Nonconvex Potentials , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Seungyong Lee,et al.  Video deblurring for hand-held cameras using patch-based synthesis , 2012, ACM Trans. Graph..

[11]  Hongxun Yao,et al.  Text image deblurring via two-tone prior , 2017, Neurocomputing.

[12]  Sunghyun Cho,et al.  Edge-based blur kernel estimation using patch priors , 2013, IEEE International Conference on Computational Photography (ICCP).

[13]  Jiri Matas,et al.  DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Sunghyun Cho,et al.  Fast motion deblurring , 2009, SIGGRAPH 2009.

[15]  Qiheng Zhang,et al.  Blind deconvolution of a noisy degraded image. , 2009, Applied optics.

[16]  Ming-Hsuan Yang,et al.  Deblurring Face Images with Exemplars , 2014, ECCV.

[17]  Yi Gao,et al.  Image restoration using spatially variant hyper-Laplacian prior , 2018, Signal Image Video Process..

[18]  Li Xu,et al.  Unnatural L0 Sparse Representation for Natural Image Deblurring , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Laure Blanc-Féraud,et al.  Sparse Poisson Noisy Image Deblurring , 2012, IEEE Transactions on Image Processing.

[20]  Frédo Durand,et al.  Understanding Blind Deconvolution Algorithms , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Xiang Zhu,et al.  Deconvolving PSFs for a Better Motion Deblurring Using Multiple Images , 2012, ECCV.

[22]  Ming-Hsuan Yang,et al.  Deblurring Text Images via L0-Regularized Intensity and Gradient Prior , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Jian Sun,et al.  Progressive inter-scale and intra-scale non-blind image deconvolution , 2008, SIGGRAPH 2008.

[24]  P. E. Robinson,et al.  Blind deconvolution of Gaussian blurred images containing additive white Gaussian noise , 2013, 2013 IEEE International Conference on Industrial Technology (ICIT).

[25]  Deqing Sun,et al.  Blind Image Deblurring Using Dark Channel Prior , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Ming-Hsuan Yang,et al.  Deblurring Low-Light Images with Light Streaks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Jean Ponce,et al.  Non-uniform Deblurring for Shaken Images , 2012, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  M. Ng,et al.  A nonlocal total variation model for image decomposition: Illumination and reflectance , 2014 .

[29]  Christopher Zach,et al.  Multiplicative vs. Additive Half-Quadratic Minimization for Robust Cost Optimization , 2018, BMVC.

[30]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, SIGGRAPH 2007.

[31]  Xiaochun Cao,et al.  Image Deblurring via Enhanced Low-Rank Prior , 2016, IEEE Transactions on Image Processing.

[32]  Bernhard Schölkopf,et al.  Recording and Playback of Camera Shake: Benchmarking Blind Deconvolution with a Real-World Database , 2012, ECCV.

[33]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.

[34]  Ming-Hsuan Yang,et al.  Robust Kernel Estimation with Outliers Handling for Image Deblurring , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Sylvain Paris,et al.  Handling Noise in Single Image Deblurring Using Directional Filters , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Mingkui Tan,et al.  Blind Image Deconvolution by Automatic Gradient Activation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Juntao Guan,et al.  Local and Nonlocal Steering Kernel Weighted Total Variation Model for Image Denoising , 2019, Symmetry.

[38]  Seungyong Lee,et al.  Convergence Analysis of MAP Based Blur Kernel Estimation , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[39]  Stephen Lin,et al.  Motion-aware noise filtering for deblurring of noisy and blurry images , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Keigo Hirakawa,et al.  Blind Deblurring and Denoising of Images Corrupted by Unidirectional Object Motion Blur and Sensor Noise , 2016, IEEE Transactions on Image Processing.

[41]  Tae Hyun Kim,et al.  Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Richard Hartley,et al.  Phase-Only Image Based Kernel Estimation for Single Image Blind Deblurring , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Frédo Durand,et al.  Understanding and evaluating blind deconvolution algorithms , 2009, CVPR.

[44]  Wolfgang Heidrich,et al.  Stochastic Blind Motion Deblurring , 2015, IEEE Transactions on Image Processing.

[45]  Ming-Hsuan Yang,et al.  Soft-Segmentation Guided Object Motion Deblurring , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Hongdong Li,et al.  Deep Stacked Hierarchical Multi-Patch Network for Image Deblurring , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Xiaochun Cao,et al.  Image Deblurring via Extreme Channels Prior , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Hua Zhang,et al.  A convex nonlocal total variation regularization algorithm for multiplicative noise removal , 2019, EURASIP J. Image Video Process..