An additive convolution model for fast restoration of nonuniform blurred images

Nonuniform blurring would be introduced during imaging by many inevitable factors, such as defocus, camera shake, or motion. Fast restoration of nonuniform blurred images, however, remains a challenging problem. The sparse blur matrix-based approach models nonuniform blurring as the multiplication of a high-dimensional sparse blur matrix and an image vector, and suffers from the high computational and memory complexity problems. To tackle these, we propose an additive convolution model (ACM) which models nonuniform blurring as the space variant weighted sum of the convolution images of a set of basis filters. We further propose a principal component analysis-based method to learn the basis filters and weight matrices. Finally, we incorporate ACM with the total variation-based restoration model, and adopt the generalized accelerated proximal gradient algorithm for the restoration of nonuniform blurred images. Numerical results show that the proposed method is effective for the restoration of nonuniform blurred images caused by defocus or camera shake, and is superior to the sparse matrix-based approach in terms of computational and memory complexity.

[1]  Frédo Durand,et al.  Understanding Blind Deconvolution Algorithms , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Ankit Gupta,et al.  Single Image Deblurring Using Motion Density Functions , 2010, ECCV.

[3]  Subhasis Chaudhuri,et al.  An MRF Model-Based Approach to Simultaneous Recovery of Depth and Restoration from Defocused Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[5]  Stefano Soatto,et al.  A geometric approach to shape from defocus , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[7]  Bahadir K. Gunturk,et al.  Image Restoration , 2012 .

[8]  Michael S. Brown,et al.  Richardson-Lucy Deblurring for Scenes under a Projective Motion Path , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Saso Tomazic,et al.  Angle Estimation of Simultaneous Orthogonal Rotations from 3D Gyroscope Measurements , 2011, Sensors.

[10]  Jean Ponce,et al.  Non-uniform Deblurring for Shaken Images , 2012, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[12]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[13]  Michael K. Ng,et al.  A fast minimization method for blur and multiplicative noise removal , 2013, Int. J. Comput. Math..

[14]  Andrea Cavallaro,et al.  Accurate and Efficient Method for Smoothly Space-Variant Gaussian Blurring , 2010, IEEE Transactions on Image Processing.

[15]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[16]  Jiaya Jia,et al.  High-quality motion deblurring from a single image , 2008, ACM Trans. Graph..

[17]  Yiqiu Dong,et al.  Spatially dependent regularization parameter selection in total generalized variation models for image restoration , 2013, Int. J. Comput. Math..

[18]  Ming-Hsuan Yang,et al.  Fast Non-uniform Deblurring using Constrained Camera Pose Subspace , 2012, BMVC.

[19]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[20]  Wangmeng Zuo,et al.  A Generalized Accelerated Proximal Gradient Approach for Total-Variation-Based Image Restoration , 2011, IEEE Transactions on Image Processing.

[21]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, ACM Trans. Graph..

[22]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[23]  Ke Chen,et al.  Introduction to variational image-processing models and applications , 2013, Int. J. Comput. Math..

[24]  Stefano Soatto,et al.  3-D Shape Estimation and Image Restoration - Exploiting Defocus and Motion Blur , 2006 .

[25]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[26]  Michael S. Brown,et al.  Nonlinear camera response functions and image deblurring , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Stefano Soatto,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE IEEE TRANSACTION OF PATTERN RECO , 2022 .

[28]  Orly Yadid-Pecht,et al.  Quaternion Structural Similarity: A New Quality Index for Color Images , 2012, IEEE Transactions on Image Processing.

[29]  José M. Bioucas-Dias,et al.  Fast Image Recovery Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[30]  José M. Bioucas-Dias,et al.  A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration , 2007, IEEE Transactions on Image Processing.

[31]  Subhasis Chaudhuri,et al.  Depth estimation and image restoration using defocused stereo pairs , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Frédo Durand,et al.  Understanding and evaluating blind deconvolution algorithms , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Kim-Chuan Toh,et al.  An Accelerated Proximal Gradient Algorithm for Frame-Based Image Restoration via the Balanced Approach , 2011, SIAM J. Imaging Sci..

[34]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[35]  Subhasis Chaudhuri,et al.  Depth From Defocus: A Real Aperture Imaging Approach , 1999, Springer New York.

[36]  Stefano Soatto,et al.  Observing Shape from Defocused Images , 2004, International Journal of Computer Vision.

[37]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.