Deep Learning for Handling Kernel/model Uncertainty in Image Deconvolution

Most existing non-blind image deconvolution methods assume that the given blurring kernel is error-free. In practice, blurring kernel often is estimated via some blind deblurring algorithm which is not exactly the truth. Also, the convolution model is only an approximation to practical blurring effect. It is known that non-blind deconvolution is susceptible to such a kernel/model error. Based on an error-in-variable (EIV) model of image blurring that takes kernel error into consideration, this paper presents a deep learning method for deconvolution, which unrolls a total-least-squares (TLS) estimator whose relating priors are learned by neural networks (NNs). The experiments showed that the proposed method is robust to kernel/model error. It noticeably outperformed existing solutions when deblurring images using noisy kernels, e.g. the ones estimated from existing blind motion deblurring methods.

[1]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[2]  Stefan Roth,et al.  Shrinkage Fields for Effective Image Restoration , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Deqing Sun,et al.  Blind Image Deblurring Using Dark Channel Prior , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Seungyong Lee,et al.  Handling outliers in non-blind image deconvolution , 2011, 2011 International Conference on Computer Vision.

[5]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Li Xu,et al.  Unnatural L0 Sparse Representation for Natural Image Deblurring , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Matthias Zwicker,et al.  Deep Mean-Shift Priors for Image Restoration , 2017, NIPS.

[8]  David Zhang,et al.  Partial Deconvolution With Inaccurate Blur Kernel , 2018, IEEE Transactions on Image Processing.

[9]  Karen O. Egiazarian,et al.  BM3D Frames and Variational Image Deblurring , 2011, IEEE Transactions on Image Processing.

[10]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Carsten Rother,et al.  Learning to Push the Limits of Efficient FFT-Based Image Deconvolution , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[12]  Michael Möller,et al.  Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[13]  Sundaresh Ram,et al.  Removing Camera Shake from a Single Photograph , 2009 .

[14]  Rob Fergus,et al.  Blind deconvolution using a normalized sparsity measure , 2011, CVPR 2011.

[15]  Laure Blanc-Féraud,et al.  Sparse Poisson Noisy Image Deblurring , 2012, IEEE Transactions on Image Processing.

[16]  Bernhard Schölkopf,et al.  A Machine Learning Approach for Non-blind Image Deconvolution , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Li Xu,et al.  Two-Phase Kernel Estimation for Robust Motion Deblurring , 2010, ECCV.

[18]  Daniele Perrone,et al.  Total Variation Blind Deconvolution: The Devil Is in the Details , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Michael J. Black,et al.  Fields of Experts: a framework for learning image priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[20]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[21]  Narendra Ahuja,et al.  A Comparative Study for Single Image Blind Deblurring , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Frédo Durand,et al.  Efficient marginal likelihood optimization in blind deconvolution , 2011, CVPR 2011.

[23]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[24]  Michal Irani,et al.  Blind Deblurring Using Internal Patch Recurrence , 2014, ECCV.

[25]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[27]  Gene H. Golub,et al.  An analysis of the total least squares problem , 1980, Milestones in Matrix Computation.

[28]  Andrew Zisserman,et al.  Deblurring shaken and partially saturated images , 2011, ICCV Workshops.

[29]  Seungyong Lee,et al.  Fast motion deblurring , 2009, ACM Trans. Graph..

[30]  Rynson W. H. Lau,et al.  Learning Fully Convolutional Networks for Iterative Non-blind Deconvolution , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  David Zhang,et al.  Simultaneous Fidelity and Regularization Learning for Image Restoration , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Ayan Chakrabarti,et al.  A Neural Approach to Blind Motion Deblurring , 2016, ECCV.

[33]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[34]  Donald Geman,et al.  Nonlinear image recovery with half-quadratic regularization , 1995, IEEE Trans. Image Process..

[35]  Deqing Sun,et al.  Learning Data Terms for Non-blind Deblurring , 2018, ECCV.

[36]  Sunghyun Cho,et al.  Edge-based blur kernel estimation using patch priors , 2013, IEEE International Conference on Computational Photography (ICCP).

[37]  Surya Ganguli,et al.  Exact solutions to the nonlinear dynamics of learning in deep linear neural networks , 2013, ICLR.

[38]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.

[39]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[41]  Stefan Roth,et al.  Noise-Blind Image Deblurring , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Frédo Durand,et al.  Understanding and evaluating blind deconvolution algorithms , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Ce Liu,et al.  Deep Convolutional Neural Network for Image Deconvolution , 2014, NIPS.

[44]  Kang Wang,et al.  Robust Image Deblurring With an Inaccurate Blur Kernel , 2012, IEEE Transactions on Image Processing.

[45]  Sebastian Nowozin,et al.  Cascades of Regression Tree Fields for Image Restoration , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Raymond J. Carroll,et al.  Measurement error in nonlinear models: a modern perspective , 2006 .

[47]  Wei Liu,et al.  Deep Non-Blind Deconvolution via Generalized Low-Rank Approximation , 2018, NeurIPS.

[48]  Jian-Feng Cai,et al.  Blind motion deblurring from a single image using sparse approximation , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  A. N. Rajagopalan,et al.  Non-blind Deblurring: Handling Kernel Uncertainty with CNNs , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.