Multi-scale Convolutional Neural Networks for Non-blind Image Deconvolution

Image deconvolution appears in many image-related problems. Previous works tried to train neural networks directly on blurry/clean pairs to restore clean images but failed. In this work, we propose a novel neural network, trained end-to-end, pixels-to-pixels, to deblur images from blurry ones. Our key insight is to build multi-scale convolutional neural networks that extract various scale feature maps which is essential for recovering sharp images and removing artifacts. The networks take input image of arbitrary size and produce output within efficient time. We demonstrate that our approach yields better result than the state-of-the-art deconvolution algorithms on a large dataset.

[1]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[2]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[3]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, SIGGRAPH 2007.

[4]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[6]  Sebastian Nowozin,et al.  Discriminative Non-blind Deblurring , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Li Xu,et al.  Two-Phase Kernel Estimation for Robust Motion Deblurring , 2010, ECCV.

[8]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[9]  Karen O. Egiazarian,et al.  BM3D Frames and Variational Image Deblurring , 2011, IEEE Transactions on Image Processing.

[10]  William T. Freeman,et al.  Removing camera shake from a single photograph , 2006, SIGGRAPH 2006.

[11]  Rob Fergus,et al.  Restoring an Image Taken through a Window Covered with Dirt or Rain , 2013, 2013 IEEE International Conference on Computer Vision.

[12]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Sunghyun Cho,et al.  Fast motion deblurring , 2009, SIGGRAPH 2009.

[15]  Enhong Chen,et al.  Image Denoising and Inpainting with Deep Neural Networks , 2012, NIPS.

[16]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[17]  Karen O. Egiazarian,et al.  Image restoration by sparse 3D transform-domain collaborative filtering , 2008, Electronic Imaging.

[18]  Bernhard Schölkopf,et al.  A Machine Learning Approach for Non-blind Image Deconvolution , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Michael J. Black,et al.  Fields of Experts: a framework for learning image priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[20]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[21]  Ce Liu,et al.  Deep Convolutional Neural Network for Image Deconvolution , 2014, NIPS.