Defocus deblurring: a designed deep model based on CNN

Abstract. With the flourishing development of deep learning in the field of computer vision, research of defocus deblurring based on it has gradually become a hotspot. However, most of the research focuses on defocus region detection or defocus map estimation, and algorithms for directly generating restoration images are less studied. Stressing on the problems of defocus deblurring, we propose a defocus deblurring deep model based on multi-scale information and convolution neural network. Concretely, we first perform an efficient and concise multi-scale information fusion by the selective receptive field module, thus the model can adapt to the scale sensitivity of the image defocusing region. We then use the residual channel attention module in the bottleneck module to extract the correlation features between channels, which enhances the effective channels and suppress the useless ones. Finally, a fusion objective function of edge loss and mean square loss is proposed to enhance the edge details of the image. Experimental results on a large-scale defocus deblurring dual-pixel dataset demonstrate that the proposed model has better performance than the traditional and existing deep-based methods. Comparing with the methods of the state of the art, the proposed model has a 0.44-DB improvement in PSNR metric.

[1]  Elad Hazan,et al.  The Limits of Learning with Missing Data , 2016, NIPS.

[2]  Jian Yang,et al.  Selective Kernel Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Tae Hyun Kim,et al.  Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  K. Siddaraju,et al.  DIGITAL IMAGE RESTORATION , 2011 .

[5]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Chi-Man Pun,et al.  Defocus Blur Detection via Depth Distillation , 2020, ECCV.

[7]  Yi Wang,et al.  Scale-Recurrent Network for Deep Image Deblurring , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  Claudio Rosito Jung,et al.  Edge-Based Defocus Blur Estimation With Adaptive Scale Selection , 2018, IEEE Transactions on Image Processing.

[9]  Yunhong Wang,et al.  Receptive Field Block Net for Accurate and Fast Object Detection , 2017, ECCV.

[10]  Jürgen Schmidhuber,et al.  Highway Networks , 2015, ArXiv.

[11]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[12]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[15]  Yuhao Wang,et al.  MF-LRTC: Multi-filters guided low-rank tensor coding for image restoration , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[16]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[17]  D. A. Fish,et al.  Blind deconvolution by means of the Richardson-Lucy algorithm. , 1995 .

[18]  Banshidhar Majhi,et al.  Motion blur parameters estimation for image restoration , 2014 .

[19]  Enrico Magli,et al.  Image Denoising with Graph-Convolutional Neural Networks , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[20]  Jian Yang,et al.  MemNet: A Persistent Memory Network for Image Restoration , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[21]  Tomio Goto,et al.  Blind image restoration based on total variation regularization and shock filter for blurred images , 2014, 2014 IEEE International Conference on Consumer Electronics (ICCE).

[22]  M. S. Brown,et al.  Learning to Reduce Defocus Blur by Realistically Modeling Dual-Pixel Data , 2020, IEEE International Conference on Computer Vision.

[23]  Huchuan Lu,et al.  Defocus Blur Detection via Multi-Stream Bottom-Top-Bottom Network , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2008, Texts in Computer Science.

[25]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[26]  Sungkil Lee,et al.  Deep Defocus Map Estimation Using Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Yu-Bin Yang,et al.  Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections , 2016, NIPS.

[28]  Ying Chen,et al.  Single image super-resolution based on a modified U-net with mixed gradient loss , 2019, Signal, Image and Video Processing.

[29]  In-So Kweon,et al.  A Unified Approach of Multi-scale Deep and Hand-Crafted Features for Defocus Estimation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Jichang Guo,et al.  Attention Network for Non-Uniform Deblurring , 2020, IEEE Access.

[31]  Albert Y. Zomaya,et al.  DeFusionNET: Defocus Blur Detection via Recurrently Fusing and Refining Discriminative Multi-Scale Deep Features , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Pichao Wang,et al.  BR$^2$Net: Defocus Blur Detection Via a Bidirectional Channel Attention Residual Refining Network , 2021, IEEE Transactions on Multimedia.

[33]  Mohd. Junedul Haque.  A Brief Review of Image Restoration Techniques , 2016 .

[34]  H. Sebastian Seung,et al.  Natural Image Denoising with Convolutional Networks , 2008, NIPS.

[35]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.

[38]  Michael S. Brown,et al.  Defocus Deblurring Using Dual-Pixel Data , 2020, ECCV.