Deep Wiener Deconvolution: Wiener Meets Deep Learning for Image Deblurring

We present a simple and effective approach for non-blind image deblurring, combining classical techniques and deep learning. In contrast to existing methods that deblur the image directly in the standard image space, we propose to perform an explicit deconvolution process in a feature space by integrating a classical Wiener deconvolution framework with learned deep features. A multi-scale feature refinement module then predicts the deblurred image from the deconvolved deep features, progressively recovering detail and small-scale structures. The proposed model is trained in an end-to-end manner and evaluated on scenarios with both simulated and real-world image blur. Our extensive experimental results show that the proposed deep Wiener deconvolution network facilitates deblurred results with visibly fewer artifacts. Moreover, our approach quantitatively outperforms state-of-the-art non-blind image deblurring methods by a wide margin.

[1]  David Zhang,et al.  Simultaneous Fidelity and Regularization Learning for Image Restoration , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Yanning Zhang,et al.  Learning Deep Gradient Descent Optimization for Image Deconvolution , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Tommi S. Jaakkola,et al.  Towards Robust, Locally Linear Deep Networks , 2019, ICLR.

[4]  Xiaoyong Shen,et al.  Dynamic Scene Deblurring With Parameter Selective Sharing and Nested Skip Connections , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Xiaohan Chen,et al.  Plug-and-Play Methods Provably Converge with Properly Trained Denoisers , 2019, ICML.

[6]  Hongdong Li,et al.  Deep Stacked Hierarchical Multi-Patch Network for Image Deblurring , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Deqing Sun,et al.  Deblurring Images via Dark Channel Prior , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Deqing Sun,et al.  Learning Data Terms for Non-blind Deblurring , 2018, ECCV.

[9]  Mauricio Delbracio,et al.  Modeling Realistic Degradations in Non-Blind Deconvolution , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[10]  A. N. Rajagopalan,et al.  Non-blind Deblurring: Handling Kernel Uncertainty with CNNs , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Yi Wang,et al.  Scale-Recurrent Network for Deep Image Deblurring , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[12]  Jonathan T. Barron,et al.  Burst Denoising with Kernel Prediction Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[13]  Jiri Matas,et al.  DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Carsten Rother,et al.  Learning to Push the Limits of Efficient FFT-Based Image Deconvolution , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[15]  Ming-Hsuan Yang,et al.  Learning Discriminative Data Fitting Functions for Blind Image Deblurring , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Ming-Hsuan Yang,et al.  Blind Image Deblurring with Outlier Handling , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[17]  Stefan Roth,et al.  Noise-Blind Image Deblurring , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Seungyong Lee,et al.  Fast non-blind deconvolution via regularized residual networks with long/short skip-connections , 2017, 2017 IEEE International Conference on Computational Photography (ICCP).

[19]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Feng Liu,et al.  Video Frame Interpolation via Adaptive Convolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Lei Zhang,et al.  Waterloo Exploration Database: New Challenges for Image Quality Assessment Models , 2017, IEEE Transactions on Image Processing.

[22]  Stephen P. Boyd,et al.  Dirty Pixels: Optimizing Image Classification Architectures for Raw Sensor Data , 2017, ArXiv.

[23]  Rynson W. H. Lau,et al.  Learning Fully Convolutional Networks for Iterative Non-blind Deconvolution , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Michael Elad,et al.  The Little Engine That Could: Regularization by Denoising (RED) , 2016, SIAM J. Imaging Sci..

[25]  Ming-Hsuan Yang,et al.  Robust Kernel Estimation with Outliers Handling for Image Deblurring , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Narendra Ahuja,et al.  A Comparative Study for Single Image Blind Deblurring , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Deqing Sun,et al.  Blind Image Deblurring Using Dark Channel Prior , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Yu-Bin Yang,et al.  Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections , 2016, NIPS.

[29]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Sebastian Nowozin,et al.  Cascades of Regression Tree Fields for Image Restoration , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Wei Yu,et al.  On learning optimized reaction diffusion processes for effective image restoration , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[33]  Ce Liu,et al.  Deep Convolutional Neural Network for Image Deconvolution , 2014, NIPS.

[34]  Michal Irani,et al.  Blind Deblurring Using Internal Patch Recurrence , 2014, ECCV.

[35]  Stefan Roth,et al.  Shrinkage Fields for Effective Image Restoration , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Razvan Pascanu,et al.  On the Number of Linear Regions of Deep Neural Networks , 2014, NIPS.

[37]  Michael S. Brown,et al.  Nonlinear Camera Response Functions and Image Deblurring: Theoretical Analysis and Practice , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Bernhard Schölkopf,et al.  A Machine Learning Approach for Non-blind Image Deconvolution , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Sunghyun Cho,et al.  Edge-based blur kernel estimation using patch priors , 2013, IEEE International Conference on Computational Photography (ICCP).

[40]  Zohair Al-Ameen,et al.  Reducing the gaussian blur artifact from ct medical images by employing a combination of sharpening filters and iterative deblurring algorithms , 2012 .

[41]  Sebastian Nowozin,et al.  Loss-Specific Training of Non-Parametric Image Restoration Models: A New State of the Art , 2012, ECCV.

[42]  Karen O. Egiazarian,et al.  BM3D Frames and Variational Image Deblurring , 2011, IEEE Transactions on Image Processing.

[43]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[44]  Seungyong Lee,et al.  Handling outliers in non-blind image deconvolution , 2011, 2011 International Conference on Computer Vision.

[45]  Stefan Roth,et al.  Bayesian deblurring with integrated noise estimation , 2011, CVPR 2011.

[46]  Andrew Zisserman,et al.  Deblurring shaken and partially saturated images , 2011, ICCV Workshops.

[47]  Ankit Gupta,et al.  Single Image Deblurring Using Motion Density Functions , 2010, ECCV.

[48]  Li Xu,et al.  Two-Phase Kernel Estimation for Robust Motion Deblurring , 2010, ECCV.

[49]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.

[50]  Seungyong Lee,et al.  Fast motion deblurring , 2009, ACM Trans. Graph..

[51]  Frédo Durand,et al.  Understanding and evaluating blind deconvolution algorithms , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Jiaya Jia,et al.  High-quality motion deblurring from a single image , 2008, ACM Trans. Graph..

[53]  Harry Shum,et al.  Progressive inter-scale and intra-scale non-blind image deconvolution , 2008, ACM Trans. Graph..

[54]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[55]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[56]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, ACM Trans. Graph..

[57]  Michael J. Black,et al.  Fields of Experts: a framework for learning image priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[58]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[59]  William H. Richardson,et al.  Bayesian-Based Iterative Method of Image Restoration , 1972 .

[60]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series, with Engineering Applications , 1949 .