Single Image Non-uniform Blur Kernel Estimation via Adaptive Basis Decomposition

Characterizing and removing motion blur caused by camera shake or object motion remains an important task for image restoration. In recent years, removal of motion blur in photographs has seen impressive progress in the hands of deep learning-based methods, trained to map directly from blurry to sharp images. Characterization of motion blur, on the other hand, has received less attention and progress in model-based methods for restoration lags behind that of data-driven end-to-end approaches. In this paper, we propose a general, non-parametric model for dense non-uniform motion blur estimation. Given a blurry image, we estimate a set of adaptive basis kernels as well as the mixing coefficients at pixel level, producing a per-pixel map of motion blur. This rich but efficient forward model of the degradation process allows the utilization of existing tools for solving inverse problems. We show that our method overcomes the limitations of existing non-uniform motion blur estimation and that it contributes to bridging the gap between model-based and data-driven approaches for deblurring real photographs.

[1]  Zhangyang Wang,et al.  DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[2]  Bolei Zhou,et al.  Scene Parsing through ADE20K Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Luc Van Gool,et al.  Plug-and-Play Image Restoration With Deep Denoiser Prior , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Tae Hyun Kim,et al.  Dynamic Scene Deblurring , 2013, 2013 IEEE International Conference on Computer Vision.

[5]  Narendra Ahuja,et al.  A Comparative Study for Single Image Blind Deblurring , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Tae Hyun Kim,et al.  Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Tae Hyun Kim,et al.  Generalized video deblurring for dynamic scenes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Sunghyun Cho,et al.  Real-World Blur Dataset for Learning and Benchmarking Deblurring Algorithms , 2020, ECCV.

[9]  Jonathan T. Barron,et al.  Burst Denoising with Kernel Prediction Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Joost van de Weijer,et al.  Self-supervised blur detection from synthetically blurred scenes , 2019, Image Vis. Comput..

[11]  Ankit Gupta,et al.  Single Image Deblurring Using Motion Density Functions , 2010, ECCV.

[12]  Georgios D. Evangelidis,et al.  Parametric Image Alignment Using Enhanced Correlation Coefficient Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[14]  Ian D. Reid,et al.  From Motion Blur to Motion Flow: A Deep Learning Solution for Removing Heterogeneous Motion Blur , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Michael S. Brown,et al.  Richardson-Lucy Deblurring for Scenes under a Projective Motion Path , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Li Xu,et al.  Discriminative Blur Detection Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  J. Atkinson,et al.  The effect of blur on cortical responses to global form and motion. , 2015, Journal of vision.

[18]  Zhou Wang,et al.  Deep Blur Mapping: Exploiting High-Level Semantics by Deep Neural Networks , 2016, IEEE Transactions on Image Processing.

[19]  Michaël Gharbi,et al.  Basis Prediction Networks for Effective Burst Denoising With Large Kernels , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Yi Wang,et al.  Scale-Recurrent Network for Deep Image Deblurring , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Rynson W. H. Lau,et al.  Dynamic Scene Deblurring Using Spatially Variant Recurrent Neural Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[22]  Ying Wu,et al.  Motion from blur , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Guillermo Sapiro,et al.  Deep Video Deblurring for Hand-Held Cameras , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Ming-Hsuan Yang,et al.  Deblurring Text Images via L0-Regularized Intensity and Gradient Prior , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Lina J. Karam,et al.  Spatially-Varying Blur Detection Based on Multiscale Fused and Sorted Transform Coefficients of Gradient Magnitudes , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Radu Timofte,et al.  NTIRE 2019 Challenge on Video Deblurring and Super-Resolution: Dataset and Study , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[27]  Seungyong Lee,et al.  Handling outliers in non-blind image deconvolution , 2011, 2011 International Conference on Computer Vision.

[28]  Antoine Dupret,et al.  A physiological camera shake model for image stabilization systems , 2011, 2011 IEEE SENSORS Proceedings.

[29]  Yochai Blau,et al.  The Perception-Distortion Tradeoff , 2017, CVPR.

[30]  Feng Liu,et al.  Video Frame Interpolation via Adaptive Separable Convolution , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[31]  Hongdong Li,et al.  Deep Stacked Hierarchical Multi-Patch Network for Image Deblurring , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Jiri Matas,et al.  DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[35]  Ming-Hsuan Yang,et al.  Soft-Segmentation Guided Object Motion Deblurring , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Jan Kautz,et al.  Reblur2Deblur: Deblurring videos via self-supervised learning , 2018, 2018 IEEE International Conference on Computational Photography (ICCP).

[37]  T. L. Harrington,et al.  Perception of motion using blur pattern information in the moderate and high-velocity domains of vision. , 1981, Acta psychologica.

[38]  Jean Ponce,et al.  Non-uniform Deblurring for Shaken Images , 2012, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[39]  Tae Hyun Kim,et al.  Segmentation-Free Dynamic Scene Deblurring , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Michael Elad,et al.  The Little Engine That Could: Regularization by Denoising (RED) , 2016, SIAM J. Imaging Sci..

[41]  Feng Liu,et al.  Video Frame Interpolation via Adaptive Convolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Guillermo Sapiro,et al.  Removing Camera Shake via Weighted Fourier Burst Accumulation , 2015, IEEE Transactions on Image Processing.

[43]  Bernhard Schölkopf,et al.  Fast removal of non-uniform camera shake , 2011, 2011 International Conference on Computer Vision.

[44]  Jean Ponce,et al.  Learning a convolutional neural network for non-uniform motion blur removal , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Li Xu,et al.  Two-Phase Kernel Estimation for Robust Motion Deblurring , 2010, ECCV.

[46]  Michael Elad,et al.  Regularization by Denoising via Fixed-Point Projection (RED-PRO) , 2020, SIAM J. Imaging Sci..

[47]  Maren Bennewitz,et al.  Efficient Vision-based Navigation Learning about the Influence of Motion Blur , 2010 .

[48]  Luc Van Gool,et al.  Dynamic Filter Networks , 2016, NIPS.

[49]  Jun Ma,et al.  Depth from zooming , 1990 .

[50]  Sundaresh Ram,et al.  Removing Camera Shake from a Single Photograph , 2009 .

[51]  Seungyong Lee,et al.  Fast motion deblurring , 2009, ACM Trans. Graph..

[52]  Bernhard Schölkopf,et al.  Recording and Playback of Camera Shake: Benchmarking Blind Deconvolution with a Real-World Database , 2012, ECCV.

[53]  Stefano Soatto,et al.  A variational approach to scene reconstruction and image segmentation from motion-blur cues , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..