Blind Motion Deblurring Based on Generative Adversarial Networks

In the past two years, GAN (Generative Adversarial Networks) has emerged and been applied to the image deblurring problem, showing good results, especially in restoring high-frequency texture details of the image. However, there are very few papers on GAN-based image deblurring so far, and it is not ideal for the processing of edge features. In this paper, an end-to-end blind image motion deblurring algorithm based on GAN is proposed. In the pixel domain, we use the weighted sum of cross entropy and L1 loss as the loss function. In the feature domain, the weighted sum of the features extracted by VGG and DenseNet is used to calculate the loss. And we add "deconvolution + PixelShuffle" module to the network. Experiments show that our method achieves the excellent performance in terms of PSNR (Peak Signal to Noise Ratio) and SSIM (Structural Similarity Index), simultaneously eliminates the checkerboards effectively.

[1]  Johannes R. Sveinsson,et al.  MTF-Based Deblurring Using a Wiener Filter for CS and MRA Pansharpening Methods , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[2]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[3]  Jia Deng,et al.  A large-scale hierarchical image database , 2009, CVPR 2009.

[4]  Jiri Matas,et al.  DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Tae Hyun Kim,et al.  Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Yoshua Bengio,et al.  Generative Adversarial Networks , 2014, ArXiv.

[7]  Dimitris N. Metaxas,et al.  StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[10]  Jean Ponce,et al.  Learning a convolutional neural network for non-uniform motion blur removal , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Bernhard Schölkopf,et al.  A Machine Learning Approach for Non-blind Image Deconvolution , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[13]  Sundaresh Ram,et al.  Removing Camera Shake from a Single Photograph , 2009 .

[14]  François-Xavier Le Dimet,et al.  Deblurring From Highly Incomplete Measurements for Remote Sensing , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Michal Hradiš,et al.  CNN for license plate motion deblurring , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[16]  Lei Bai,et al.  Generative Adversarial Network for Deblurring of Remote Sensing Image , 2018, 2018 26th International Conference on Geoinformatics.

[17]  Seungyong Lee,et al.  Handling outliers in non-blind image deconvolution , 2011, 2011 International Conference on Computer Vision.

[18]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Aggelos K. Katsaggelos,et al.  Tracking Motion-Blurred Targets in Video , 2006, 2006 International Conference on Image Processing.

[20]  Li Xu,et al.  Forward Motion Deblurring , 2013, 2013 IEEE International Conference on Computer Vision.

[21]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Tao Qin,et al.  Conditional Image-to-Image Translation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[23]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[24]  Wei-Chang Tsai,et al.  Motion deblurring from a single photograph based on kernel estimation , 2013, 2013 9th International Conference on Information, Communications & Signal Processing.

[25]  Jian Yu,et al.  A Dictionary Learning Approach for Poisson Image Deblurring , 2013, IEEE Transactions on Medical Imaging.

[26]  Richard Szeliski,et al.  Image deblurring and denoising using color priors , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Oleg V. Michailovich,et al.  A novel approach to the 2-D blind deconvolution problem in medical ultrasound , 2005, IEEE Transactions on Medical Imaging.