Clustering of matched features and gradient matching for mixed-resolution video super-resolution

This work presents a novel technique for image reconstruction applied to mixed-resolution video super-resolution. We segment an image into patches defined by the clustering of a vector flow generated from matching SIFT features. We reconstruct the segmented image by applying image projective transformation to a reference image. By varying the number of clusters, we composed a sequence of reconstructed images, which are then used to compose a codebook, through gradient matching. This idea is extended to use low and high-resolution image pairs for super-resolution. Our results indicate a 1.4dB gain, on average, over the use of overlapped-block motion-compensation (OBMC).

[1]  Edson M. Hung,et al.  Video super-resolution based on local invariant features matching , 2012, 2012 19th IEEE International Conference on Image Processing.

[2]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Edson M. Hung,et al.  Video Super-Resolution Using Codebooks Derived From Key-Frames , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[5]  Norbert Brändle,et al.  Evaluation of clustering methods for finding dominant optical flow fields in crowded scenes , 2008, 2008 19th International Conference on Pattern Recognition.

[6]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[7]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[8]  Anastasis A. Sofokleous,et al.  Review: H.264 and MPEG-4 Video Compression: Video Coding for Next-generation Multimedia , 2005, Comput. J..

[9]  Robin Hess.  An Open-Source SIFT Library , 2010 .

[10]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2008, Texts in Computer Science.

[11]  Bruno Macchiavello,et al.  A simple reversed-complexity Wyner-Ziv video coding mode based on a spatial reduction framework , 2007, Electronic Imaging.

[12]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[13]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[14]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[15]  Iain E. G. Richardson,et al.  H.264 and MPEG-4 Video Compression: Video Coding for Next-Generation Multimedia , 2003 .

[16]  J. H. Ward.  Hierarchical Grouping to Optimize an Objective Function , 1963 .

[17]  Byung Cheol Song,et al.  Video Super-Resolution Algorithm Using Bi-Directional Overlapped Block Motion Compensation and On-the-Fly Dictionary Training , 2011, IEEE Transactions on Circuits and Systems for Video Technology.