Optimizing and Learning for Super-resolution

In multiple-image super-resolution, a high resolution image is estimated from a number of lower-resolution images. This involves computing the parameters of a generative imaging model (such as geometric and photometric registration, and blur) and obtaining a MAP estimate by minimizing a cost function including an appropriate prior. We consider the quite general geometric registration situation modelled by a plane projective transformation, and make two novel contributions: (i) in previous approaches the MAP estimate has been obtained by fir st computing and fixing the registration, and then computing the super-re solution image with this registration. We demonstrate that superior estim ates are obtained by optimizing over both the registration and image; (ii) the parameters of the edge preserving prior are learnt automatically from the data, rather than being set by trial and error. We show examples on a number of real sequences including multiple stills, digital video, and DVDs of movies.

[1]  Peyman Milanfar,et al.  Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement , 2001, IEEE Trans. Image Process..

[2]  Stephen J. Roberts,et al.  A Sampled Texture Prior for Image Super-Resolution , 2003, NIPS.

[3]  Russell M. Mersereau,et al.  Blur identification by the method of generalized cross-validation , 1992, IEEE Trans. Image Process..

[4]  Sean Borman,et al.  Topics in Multiframe Superresolution Restoration , 2004 .

[5]  David Capel,et al.  Image Mosaicing and Super-resolution , 2004, Distinguished Dissertations.

[6]  Manjunath V. Joshi,et al.  A learning-based method for image super-resolution from zoomed observations , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[7]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[8]  Harry Shum,et al.  Patch based blind image super resolution , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[9]  Ian T. Nabney,et al.  Netlab: Algorithms for Pattern Recognition , 2002 .

[10]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[11]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[12]  A. Murat Tekalp,et al.  Maximum likelihood image and blur identification: a unifying , 1990 .

[13]  Russell C. Hardie,et al.  Joint MAP registration and high-resolution image estimation using a sequence of undersampled images , 1997, IEEE Trans. Image Process..

[14]  Michael Elad,et al.  A practical approach to super-resolution , 2006 .

[15]  Michal Irani,et al.  Super resolution from image sequences , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[16]  Yücel Altunbasak,et al.  Super-resolution still and video reconstruction from MPEG-coded video , 2002, IEEE Trans. Circuits Syst. Video Technol..

[17]  Luc Van Gool,et al.  A Probabilistic Approach to Optical Flow based Super-Resolution , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[18]  Christopher M. Bishop,et al.  Bayesian Image Super-Resolution , 2002, NIPS.