Image and Video Restorations via Nonlocal Kernel Regression

A nonlocal kernel regression (NL-KR) model is presented in this paper for various image and video restoration tasks. The proposed method exploits both the nonlocal self-similarity and local structural regularity properties in natural images. The nonlocal self-similarity is based on the observation that image patches tend to repeat themselves in natural images and videos, and the local structural regularity observes that image patches have regular structures where accurate estimation of pixel values via regression is possible. By unifying both properties explicitly, the proposed NL-KR framework is more robust in image estimation, and the algorithm is applicable to various image and video restoration tasks. In this paper, we apply the proposed model to image and video denoising, deblurring, and superresolution reconstruction. Extensive experimental results on both single images and realistic video sequences demonstrate that the proposed framework performs favorably with previous works both qualitatively and quantitatively.

[1]  Laurent D. Cohen,et al.  Non-local Regularization of Inverse Problems , 2008, ECCV.

[2]  Gabriel Peyré,et al.  Manifold models for signals and images , 2009, Comput. Vis. Image Underst..

[3]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[5]  Michael Elad,et al.  Super-Resolution Without Explicit Subpixel Motion Estimation , 2009, IEEE Transactions on Image Processing.

[6]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[7]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[8]  M. Levoy,et al.  Gaussian KD-trees for fast high-dimensional filtering , 2009, SIGGRAPH 2009.

[9]  Adam Finkelstein,et al.  The Generalized PatchMatch Correspondence Algorithm , 2010, ECCV.

[10]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[11]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[12]  A. Foi,et al.  IMAGE AND VIDEO SUPER-RESOLUTION VIA SPATIALLY ADAPTIVE BLOCK-MATCHING FILTERING , 2008 .

[13]  Rama Chellappa,et al.  Example-Driven Manifold Priors for Image Deconvolution , 2011, IEEE Transactions on Image Processing.

[14]  Yanning Zhang,et al.  Sparse representation based iterative incremental image deblurring , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[15]  Thomas S. Huang,et al.  Exploiting Structured Sparsity for Image Deblurring , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[16]  Richard G. Baraniuk,et al.  ForWaRD: Fourier-wavelet regularized deconvolution for ill-conditioned systems , 2004, IEEE Transactions on Signal Processing.

[17]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[18]  Peyman Milanfar,et al.  Deblurring Using Regularized Locally Adaptive Kernel Regression , 2008, IEEE Transactions on Image Processing.

[19]  D. Tschumperlé.  PDE's based regularization of multivalued images and applications , 2002 .

[20]  Thomas S. Huang,et al.  Sparse representation based blind image deblurring , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[21]  Raanan Fattal,et al.  Image upsampling via imposed edge statistics , 2007, ACM Trans. Graph..

[22]  Xin Li,et al.  Video Processing Via Implicit and Mixture Motion Models , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  Michael Elad,et al.  Image Denoising Via Learned Dictionaries and Sparse representation , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[24]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[25]  Michael Elad,et al.  Fast and Robust Multi-Frame Super-Resolution , 2004, IEEE Transactions on Image Processing.

[26]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[27]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[28]  Thomas S. Huang,et al.  Coupled Dictionary Training for Image Super-Resolution , 2012, IEEE Transactions on Image Processing.

[29]  Peyman Milanfar,et al.  A generalization of non-local means via kernel regression , 2008, Electronic Imaging.

[30]  Dennis M. Healy,et al.  Shearlet-Based Deconvolution , 2009, IEEE Transactions on Image Processing.

[31]  Thomas S. Huang,et al.  Multi-scale Non-Local Kernel Regression for super resolution , 2011, 2011 18th IEEE International Conference on Image Processing.

[32]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[33]  Klamer Schutte,et al.  Robust Fusion of Irregularly Sampled Data Using Adaptive Normalized Convolution , 2006, EURASIP J. Adv. Signal Process..

[34]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[35]  Karen O. Egiazarian,et al.  Image restoration by sparse 3D transform-domain collaborative filtering , 2008, Electronic Imaging.

[36]  Michael Elad,et al.  Super Resolution With Probabilistic Motion Estimation , 2009, IEEE Transactions on Image Processing.

[37]  Michael Elad,et al.  Generalizing the Nonlocal-Means to Super-Resolution Reconstruction , 2009, IEEE Transactions on Image Processing.

[38]  Thomas S. Huang,et al.  Generative Bayesian Image Super Resolution With Natural Image Prior , 2012, IEEE Transactions on Image Processing.

[39]  Thomas S. Huang,et al.  Non-Local Kernel Regression for Image and Video Restoration , 2010, ECCV.

[40]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[41]  Thomas S. Huang,et al.  Close the loop: Joint blind image restoration and recognition with sparse representation prior , 2011, 2011 International Conference on Computer Vision.

[42]  A. N. Rajagopalan,et al.  Superresolution of License Plates in Real Traffic Videos , 2007, IEEE Transactions on Intelligent Transportation Systems.

[43]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[44]  Kwang In Kim,et al.  Example-Based Learning for Single-Image Super-Resolution , 2008, DAGM-Symposium.