A learning-based method for image super-resolution from zoomed observations

We propose a technique for super-resolution imaging of a scene from observations at different camera zooms. Given a sequence of images with different zoom factors of a static scene, we obtain a picture of the entire scene at a resolution corresponding to the most zoomed observation. The high-resolution image is modeled through appropriate parameterization, and the parameters are learned from the most zoomed observation. Assuming a homogeneity of the high-resolution field, the learned model is used as a prior while super-resolving the scene. We suggest the use of either a Markov random field (MRF) or an simultaneous autoregressive (SAR) model to parameterize the field based on the computation one can afford. We substantiate the suitability of the proposed method through a large number of experimentations on both simulated and real data.

[1]  Sridhar Lakshmanan,et al.  Simultaneous Parameter Estimation and Segmentation of Gibbs Random Fields Using Simulated Annealing , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Michael Elad,et al.  Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images , 1997, IEEE Trans. Image Process..

[3]  Yaron Caspi,et al.  Increasing Space-Time Resolution in Video , 2002, ECCV.

[4]  Robert L. Stevenson,et al.  Extraction of high-resolution frames from video sequences , 1996, IEEE Trans. Image Process..

[5]  Peter Cheeseman,et al.  Super-Resolved Surface Reconstruction from Multiple Images , 1996 .

[6]  Yihua Yu,et al.  MRF parameter estimation by an accelerated method , 2003, Pattern Recognit. Lett..

[7]  H. Künsch Gaussian Markov random fields , 1979 .

[8]  Michael Elad,et al.  A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur , 2001, IEEE Trans. Image Process..

[9]  Gerasimos Potamianos,et al.  Stochastic approximation algorithms for partition function estimation of Gibbs random fields , 1997, IEEE Trans. Inf. Theory.

[10]  Takeo Kanade,et al.  Limits on Super-Resolution and How to Break Them , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Yücel Altunbasak,et al.  Super-resolution still and video reconstruction from MPEG-coded video , 2002, IEEE Trans. Circuits Syst. Video Technol..

[12]  Rama Chellappa,et al.  Data-driven multichannel superresolution with application to video sequences , 1999 .

[13]  A. Murat Tekalp,et al.  High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[14]  Yücel Altunbasak,et al.  Eigenface-based super-resolution for face recognition , 2002, Proceedings. International Conference on Image Processing.

[15]  Michael Elad,et al.  Superresolution restoration of an image sequence: adaptive filtering approach , 1999, IEEE Trans. Image Process..

[16]  Heung-Yeung Shum,et al.  Fundamental limits of reconstruction-based superresolution algorithms under local translation , 2004 .

[17]  Gerasimos Potamianos,et al.  Partition function estimation of Gibbs random field images using Monte Carlo simulations , 1993, IEEE Trans. Inf. Theory.

[18]  A. Murat Tekalp,et al.  Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time , 1997, IEEE Trans. Image Process..

[19]  Robert L. Stevenson,et al.  Simultaneous multi-frame MAP super-resolution video enhancement using spatio-temporal priors , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[20]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[21]  Peyman Milanfar,et al.  A computationally efficient superresolution image reconstruction algorithm , 2001, IEEE Trans. Image Process..

[22]  Sven J. Dickinson,et al.  A quantitative analysis of view degeneracy and its use for active focal length control , 1995, Proceedings of IEEE International Conference on Computer Vision.

[23]  Nirmal K. Bose,et al.  Recursive reconstruction of high resolution image from noisy undersampled multiframes , 1990, IEEE Trans. Acoust. Speech Signal Process..

[24]  Jun Ma,et al.  Depth from zooming , 1990 .

[25]  MumfordDavid,et al.  Filters, Random Fields and Maximum Entropy (FRAME) , 1998 .

[26]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[27]  Michel Dhome,et al.  Dense Reconstruction by Zooming , 1996, ECCV.

[28]  Stan Z. Li,et al.  Markov Random Field Modeling in Computer Vision , 1995, Computer Science Workbench.

[29]  Song-Chun Zhu,et al.  Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling , 1998, International Journal of Computer Vision.

[30]  Aggelos K. Katsaggelos,et al.  High-resolution image reconstruction of low-resolution DCT-based compressed images , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[31]  Michael K. Ng,et al.  Constrained total least‐squares computations for high‐resolution image reconstruction with multisensors , 2002, Int. J. Imaging Syst. Technol..

[32]  J. Besag Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[33]  Ehud Rivlin,et al.  Zoom tracking and its applications , 2001, Machine Vision and Applications.

[34]  Anil K. Jain,et al.  Texture classification and segmentation using multiresolution simultaneous autoregressive models , 1992, Pattern Recognit..

[35]  Robert L. Stevenson,et al.  A Bayesian approach to image expansion for improved definitio , 1994, IEEE Trans. Image Process..

[36]  José Carlos Príncipe,et al.  Super-resolution of images based on local correlations , 1999, IEEE Trans. Neural Networks.

[37]  Rama Chellappa,et al.  Estimation and choice of neighbors in spatial-interaction models of images , 1983, IEEE Trans. Inf. Theory.

[38]  Alireza Khotanzad,et al.  Multispectral Random Field Models for Synthesis and Analysis of Color Images , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Daniel Gross,et al.  Improved resolution from subpixel shifted pictures , 1992, CVGIP Graph. Model. Image Process..

[40]  Aggelos K. Katsaggelos,et al.  Bayesian high-resolution reconstruction of low-resolution compressed video , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[41]  Ming-Chao Chiang,et al.  Local blur estimation and super-resolution , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[42]  Andrew Zisserman,et al.  Automated mosaicing with super-resolution zoom , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[43]  Ambasamudram N Rajagopalan,et al.  Motion-free superresolution and the role of relative blur. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[44]  Deepu Rajan,et al.  Simultaneous estimation of super-resolved intensity and depth maps from low resolution defocused observations of a scene , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[45]  Michel Dhome,et al.  Three-dimensional reconstruction by zooming , 1993, IEEE Trans. Robotics Autom..

[46]  Song-Chun Zhu,et al.  Learning in Gibbsian Fields: How Accurate and How Fast Can It Be? , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Russell C. Hardie,et al.  Joint MAP registration and high-resolution image estimation using a sequence of undersampled images , 1997, IEEE Trans. Image Process..

[48]  Anil K. Jain,et al.  Parameter estimation in MRF line process models , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[49]  Subhasis Chaudhuri,et al.  Performance Analysis of Maximum Likelihood Estimator for Recovery of Depth from Defocused Images and Optimal Selection of Camera Parameters , 1998, International Journal of Computer Vision.

[50]  Uday B. Desai,et al.  A multiresolution approach to color image restoration and parameter estimation using homotopy continuation method , 1995, Proceedings., International Conference on Image Processing.

[51]  Uday B. Desai,et al.  Compressed Domain Video Zooming: Use of Motion Vectors , 2001 .

[52]  Michal Irani,et al.  Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency , 1993, J. Vis. Commun. Image Represent..

[53]  Andrew Zisserman,et al.  Super-resolution from multiple views using learnt image models , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[54]  Deepu Rajan,et al.  An MRF-Based Approach to Generation of Super-Resolution Images from Blurred Observations , 2004, Journal of Mathematical Imaging and Vision.

[55]  S. Chaudhuri Super-Resolution Imaging , 2001 .

[56]  Song-Chun Zhu,et al.  Minimax Entropy Principle and Its Application to Texture Modeling , 1997, Neural Computation.

[57]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[58]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[59]  Terrance E. Boult,et al.  Efficient super-resolution via image warping , 2000, Image Vis. Comput..