Video upscaling via spatio-temporal self-similarity

We propose a new example-based video upscaling technique that exploits self-similarity among patches of a video in both space and time. We encode image patches with over-complete dictionaries constructed in a local spatio-temporal neighborhood, and establish temporal correspondence using modern optical flow techniques. The resulting method performs favorably compared to the state-of-the-art in super-resolution techniques.

[1]  Stefano Soatto,et al.  Direct Sparse Deblurring , 2010, Journal of Mathematical Imaging and Vision.

[2]  Michael Elad,et al.  Super-Resolution Without Explicit Subpixel Motion Estimation , 2009, IEEE Transactions on Image Processing.

[3]  Michal Irani,et al.  Super resolution from image sequences , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[4]  Stefano Soatto,et al.  Sparse Occlusion Detection with Optical Flow , 2012, International Journal of Computer Vision.

[5]  Shmuel Peleg,et al.  Robust super-resolution , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[6]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[7]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[8]  N. Paragios,et al.  A high-quality video denoising algorithm based on reliable motion estimation , 2010 .

[9]  Peyman Milanfar,et al.  Deblurring Using Regularized Locally Adaptive Kernel Regression , 2008, IEEE Transactions on Image Processing.

[10]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Michael Elad,et al.  Generalizing the Nonlocal-Means to Super-Resolution Reconstruction , 2009, IEEE Transactions on Image Processing.

[12]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[13]  Raanan Fattal,et al.  Image and video upscaling from local self-examples , 2011, TOGS.