Linear models for multiframe super-resolution restoration under nonaffine registration and spatially varying PSF

Multi-frame super-resolution restoration refers to techniques for still-image and video restoration which utilize multiple observed images of an underlying scene to achieve the restoration of super-resolved imagery. An observation model which relates the measured data to the unknowns to be estimated is formulated to account for the registration of the multiple observations to a fixed reference frame as well as for spatial and temporal degradations resulting from characteristics of the optical system, sensor system and scene motion. Linear observation models, in which the observation process is described by a linear transformation, have been widely adopted. In this paper we consider the application of the linear observation model to multi-frame super-resolution restoration under conditions of non-affine image registration and spatially varying PSF. Reviewing earlier results, we show how these conditions relate to the technique of image warping from the computer graphics literature and how these ideas may be applied to multi-frame restoration. We illustrate the application of these methods to multi-frame super-resolution restoration using a Bayesian inference framework to solve the ill-posed restoration inverse problem.

[1]  Edward J. Delp,et al.  Discontinuity preserving regularization of inverse visual problems , 1994, IEEE Trans. Syst. Man Cybern..

[2]  J. Laurie Snell,et al.  Markov Random Fields and Their Applications , 1980 .

[3]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[4]  Paul S. Heckbert,et al.  Fundamentals of Texture Mapping and Image Warping , 1989 .

[5]  P. Hansen.  Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion , 1987 .

[6]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[7]  C. M. Shetty,et al.  Nonlinear Programming - Theory and Algorithms, Second Edition , 1993 .

[8]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms , 1993 .

[9]  A.K. Krishnamurthy,et al.  Multidimensional digital signal processing , 1985, Proceedings of the IEEE.

[10]  John Odentrantz,et al.  Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues , 2000, Technometrics.

[11]  Robert L. Stevenson,et al.  Extraction of high-resolution frames from video sequences , 1996, IEEE Trans. Image Process..

[12]  Anil K. Jain,et al.  Random field models in image analysis , 1989 .

[13]  Anil K. Jain,et al.  Markov Random Field Texture Models , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Haluk Derin,et al.  Modeling and Segmentation of Noisy and Textured Images Using Gibbs Random Fields , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Robert L. Stevenson,et al.  Image resampling and constraint formulation for multiframe superresolution restoration , 2003, IS&T/SPIE Electronic Imaging.