Non-Local Kernel Regression for Image and Video Restoration

This paper presents a non-local kernel regression (NL-KR) method for image and video restoration tasks, which exploits both the non-local self-similarity and local structural regularity in natural images. The non-local self-similarity is based on the observation that image patches tend to repeat themselves in natural images and videos; and the local structural regularity reveals that image patches have regular structures where accurate estimation of pixel values via regression is possible. Explicitly unifying both properties, the proposed non-local kernel regression framework is robust and applicable to various image and video restoration tasks. In this work, we are specifically interested in applying the NL-KR model to image and video super-resolution (SR) reconstruction. Extensive experimental results on both single images and realistic video sequences demonstrate the superiority of the proposed framework for SR tasks over previous works both qualitatively and quantitatively.

[1]  Michael Elad,et al.  Image Denoising Via Learned Dictionaries and Sparse representation , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[2]  Alexander Wong,et al.  A nonlocal-means approach to exemplar-based inpainting , 2008, 2008 15th IEEE International Conference on Image Processing.

[3]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[4]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Michael Elad,et al.  Fast and Robust Multi-Frame Super-Resolution , 2004, IEEE Transactions on Image Processing.

[6]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[7]  Peyman Milanfar,et al.  A generalization of non-local means via kernel regression , 2008, Electronic Imaging.

[8]  Gabriel Peyré,et al.  Manifold models for signals and images , 2009, Comput. Vis. Image Underst..

[9]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Michael Elad,et al.  Super-Resolution Without Explicit Subpixel Motion Estimation , 2009, IEEE Transactions on Image Processing.

[11]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[12]  Laurent D. Cohen,et al.  Non-local Regularization of Inverse Problems , 2008, ECCV.

[13]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[14]  Kwang In Kim,et al.  Example-Based Learning for Single-Image Super-Resolution , 2008, DAGM-Symposium.

[15]  Michael Elad,et al.  Generalizing the Nonlocal-Means to Super-Resolution Reconstruction , 2009, IEEE Transactions on Image Processing.

[16]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[18]  D. Tschumperlé.  PDE's based regularization of multivalued images and applications , 2002 .

[19]  Rachid Deriche,et al.  Vector-valued image regularization with PDEs: a common framework for different applications , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Karen O. Egiazarian,et al.  Image upsampling via spatially adaptive block-matching filtering , 2008, 2008 16th European Signal Processing Conference.

[21]  Yanning Zhang,et al.  Sparse representation based iterative incremental image deblurring , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[22]  Peyman Milanfar,et al.  Deblurring Using Regularized Locally Adaptive Kernel Regression , 2008, IEEE Transactions on Image Processing.

[23]  Richard G. Baraniuk,et al.  The multiscale structure of non-differentiable image manifolds , 2005, SPIE Optics + Photonics.

[24]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[25]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[26]  Marc Levoy,et al.  Gaussian KD-trees for fast high-dimensional filtering , 2009, ACM Trans. Graph..

[27]  Richard G. Baraniuk,et al.  ForWaRD: Fourier-wavelet regularized deconvolution for ill-conditioned systems , 2004, IEEE Transactions on Signal Processing.

[28]  Raanan Fattal,et al.  Image upsampling via imposed edge statistics , 2007, ACM Trans. Graph..

[29]  Xin Li,et al.  Video Processing Via Implicit and Mixture Motion Models , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[31]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[32]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[33]  Michael Elad,et al.  Super Resolution With Probabilistic Motion Estimation , 2009, IEEE Transactions on Image Processing.