Video super-resolution using high quality photographs

This paper introduces a technique that increases the spatial resolution of a given video. The method is built on the fundamentals of super-resolution techniques that aim to reconstruct high-resolution frames from a low-resolution input sequence. Different than classical super-resolution methods, besides using the information of adjacent frames, we take advantage of several reference high quality (resolution) photographs of the same scene. The method is purely imagebased, and does not require depth estimation. The additional information extracted from the reference photographs is used to construct several high resolution seed frames added with a constant step in the initial video sequence. Therefore, the seed frames but also the adjacent low-resolution frames provide important information to define priors that are considered in the probabilistic interpretation of the generative model. The estimated solution is obtained based on a standard maximum a posteriori (MAP) approach. Objective tests on real and synthetic video sequences demonstrate the utility and the benefits of the proposed technique over related methods.

[1]  Robert L. Stevenson,et al.  Spatial Resolution Enhancement of Low-Resolution Image Sequences A Comprehensive Review with Directions for Future Research , 1998 .

[2]  Michael Elad,et al.  Fast and Robust Multi-Frame Super-Resolution , 2004, IEEE Transactions on Image Processing.

[3]  Cosmin Ancuti,et al.  An efficient two steps algorithm for wide baseline image matching , 2009, The Visual Computer.

[4]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[5]  Maneesh Agrawala,et al.  Using Photographs to Enhance Videos of a Static Scene , 2007, Rendering Techniques.

[6]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[7]  Cosmin Ancuti,et al.  Video enhancement using reference photographs , 2007, SIGGRAPH '07.

[8]  Zeev Farbman,et al.  Edge-preserving decompositions for multi-scale tone and detail manipulation , 2008, SIGGRAPH 2008.

[9]  Brendan J. Frey,et al.  Video Epitomes , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[11]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[12]  Stephen J. Roberts,et al.  Optimizing and Learning for Super-resolution , 2006, BMVC.

[13]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[14]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[15]  Zeev Farbman,et al.  Edge-preserving decompositions for multi-scale tone and detail manipulation , 2008, ACM Trans. Graph..