Joint blur kernel estimation and CNN for blind image restoration

Abstract Convolutional neural networks (CNN) have shown its excellent performance in computer vision fields. Recently, they are successfully applied to image restoration. This paper proposes a joint blur kernel estimation and CNN method for blind image restoration. The blur kernel estimation is based on both blur support parameter estimation and blur type identification. An automatic feature line detection algorithm is presented for blur support parameter estimation and a dictionary learning algorithm is presented for the blur type identification. Once the blur kernel estimate is obtained, we use an effective CNN for iterative non-blind deconvolution, which is able to automatically learn image priors. Compared with current blind image restoration methods, the proposed joint method can obtain restored images under three types of unknown blur kernels. The experimental result shows that the proposed blur kernel estimation algorithm can provide high accuracy results. Furthermore, the proposed joint blur kernel estimation and CNN algorithm is superior to conventional blind image restoration algorithms in terms of restoration quality and computation time.

[1]  Banshidhar Majhi,et al.  Motion blur parameters estimation for image restoration , 2014 .

[2]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[3]  Henry Leung,et al.  A discrete-time learning algorithm for image restoration using a novel L2-norm noise constrained estimation , 2016, Neurocomputing.

[4]  Youshen Xia,et al.  Blind super-resolution image reconstruction based on novel blur type identification , 2017, 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[5]  S. Sathiya Keerthi,et al.  Which Is the Best Multiclass SVM Method? An Empirical Study , 2005, Multiple Classifier Systems.

[6]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[7]  Li Chen,et al.  A soft double regularization approach to parametric blind image deconvolution , 2005, IEEE Transactions on Image Processing.

[8]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[9]  Deqing Sun,et al.  Blind Image Deblurring Using Dark Channel Prior , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Rynson W. H. Lau,et al.  Learning Fully Convolutional Networks for Iterative Non-blind Deconvolution , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Michel Barlaud,et al.  Blind restoration of noisy blurred image using a constrained maximum likelihood method , 1991 .

[12]  Jiaya Jia,et al.  Image partial blur detection and classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Xiaochun Cao,et al.  Image Deblurring via Extreme Channels Prior , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Narendra Ahuja,et al.  A Comparative Study for Single Image Blind Deblurring , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Yikang Yang,et al.  Parametric blur estimation for blind restoration of atmospherically degraded images: Class G , 2017 .

[16]  Bernhard Schölkopf,et al.  End-to-End Learning for Image Burst Deblurring , 2016, ACCV.

[17]  Lei Zhang,et al.  Canny edge detection enhancement by scale multiplication , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[19]  Sunghyun Cho,et al.  Edge-based blur kernel estimation using patch priors , 2013, IEEE International Conference on Computational Photography (ICCP).

[20]  Wenjun Zhang,et al.  Using Free Energy Principle For Blind Image Quality Assessment , 2015, IEEE Transactions on Multimedia.

[21]  Tae Hyun Kim,et al.  Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Wenbin Li,et al.  Learn to model blurry motion via directional similarity and filtering , 2018, Pattern Recognit..

[23]  Li Chen,et al.  Efficient discrete spatial techniques for blur support identification in blind image deconvolution , 2006, IEEE Trans. Signal Process..

[24]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[25]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  M. Azimi-Sadjadi Strip Kalman filtering for image restoration: new results , 1989, IEEE International Symposium on Circuits and Systems,.

[27]  Ayan Chakrabarti,et al.  A Neural Approach to Blind Motion Deblurring , 2016, ECCV.

[28]  Rob Fergus,et al.  Blind deconvolution using a normalized sparsity measure , 2011, CVPR 2011.

[29]  Chao Ren,et al.  CISRDCNN: Super-resolution of compressed images using deep convolutional neural networks , 2017, Neurocomputing.

[30]  Chunping Hou,et al.  A two-channel convolutional neural network for image super-resolution , 2018, Neurocomputing.

[31]  Jan Kotera,et al.  Convolutional Neural Networks for Direct Text Deblurring , 2015, BMVC.

[32]  Ling Shao,et al.  Blind Image Blur Estimation via Deep Learning , 2016, IEEE Transactions on Image Processing.

[33]  Michal Hradiš,et al.  CNN for license plate motion deblurring , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[34]  Jonathon A. Chambers,et al.  Wavelet transform-based noise reduction schemes to improve the noise sensitivity of the NAS-RIF algorithm for blind image deconvolution , 2000, 2000 10th European Signal Processing Conference.

[35]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[36]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[37]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Jian Zhang,et al.  Image Restoration Using Joint Statistical Modeling in a Space-Transform Domain , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[39]  Wei Wang,et al.  Design and implementation of Log-Gabor filter in fingerprint image enhancement , 2008, Pattern Recognit. Lett..

[40]  Aggelos K. Katsaggelos,et al.  Video Super-Resolution With Convolutional Neural Networks , 2016, IEEE Transactions on Computational Imaging.

[41]  Fen Chen,et al.  An Empirical Identification Method of Gaussian Blur Parameter for Image Deblurring , 2009, IEEE Transactions on Signal Processing.

[42]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[43]  Bernhard Schölkopf,et al.  Learning to Deblur , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Haibo Wang,et al.  Blur-Kernel Bound Estimation From Pyramid Statistics , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[45]  Mostafa Kaveh,et al.  A regularization approach to joint blur identification and image restoration , 1996, IEEE Trans. Image Process..

[46]  Lei Zhang,et al.  Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index , 2013, IEEE Transactions on Image Processing.

[47]  Michael Unser,et al.  Convolutional Neural Networks for Inverse Problems in Imaging: A Review , 2017, IEEE Signal Processing Magazine.

[48]  Zhongyuan Wang,et al.  Improved scheme of estimating motion blur parameters for image restoration , 2017, Digit. Signal Process..

[49]  Peyman Milanfar,et al.  Blind Deconvolution Using Alternating Maximum a Posteriori Estimation with Heavy-Tailed Priors , 2013, CAIP.

[50]  Qianqing Qin,et al.  Image Deblurring Via Combined Total Variation and Framelet , 2014, Circuits Syst. Signal Process..

[51]  Xu Zhou,et al.  Variational Dirichlet Blur Kernel Estimation , 2015, IEEE Transactions on Image Processing.

[52]  Sunghyun Cho,et al.  Good Image Priors for Non-blind Deconvolution - Generic vs. Specific , 2014, ECCV.

[53]  Rob Fergus,et al.  Restoring an Image Taken through a Window Covered with Dirt or Rain , 2013, 2013 IEEE International Conference on Computer Vision.

[54]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.

[55]  José M. Bioucas-Dias,et al.  Parametric Blur Estimation for Blind Restoration of Natural Images: Linear Motion and Out-of-Focus , 2014, IEEE Transactions on Image Processing.

[56]  Bernhard Schölkopf,et al.  A Machine Learning Approach for Non-blind Image Deconvolution , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[57]  Michael J. Black,et al.  Fields of Experts: a framework for learning image priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[58]  Michal Irani,et al.  Blind Deblurring Using Internal Patch Recurrence , 2014, ECCV.

[59]  Deepa Kundur,et al.  Blind Image Deconvolution , 2001 .

[60]  Wei Xing Zheng,et al.  Analysis and application of a novel fast algorithm for 2-D ARMA model parameter estimation , 2013, Autom..

[61]  Shijian Lu,et al.  Blurred image region detection and classification , 2011, ACM Multimedia.

[62]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[63]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  Takumi Kobayashi,et al.  BFO Meets HOG: Feature Extraction Based on Histograms of Oriented p.d.f. Gradients for Image Classification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[65]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[66]  Thierry Blu,et al.  A Novel SURE-Based Criterion for Parametric PSF Estimation , 2015, IEEE Transactions on Image Processing.

[67]  Ming-Hsuan Yang,et al.  Motion Blur Kernel Estimation via Deep Learning , 2018, IEEE Transactions on Image Processing.

[68]  Jian Sun,et al.  A blur estimation and detection method for out-of-focus images , 2016, Multimedia Tools and Applications.