Superresolution image reconstruction from a sequence of aliased imagery.

We present a superresolution image reconstruction from a sequence of aliased imagery. The subpixel shifts (displacement) among the images are unknown due to the uncontrolled natural jitter of the imager. A correlation method is utilized to estimate subpixel shifts between each low-resolution aliased image with respect to a reference image. An error-energy reduction algorithm is derived to reconstruct the high-resolution alias-free output image. The main feature of this proposed error-energy reduction algorithm is that we treat the spatial samples from low-resolution images that possess unknown and irregular (uncontrolled) subpixel shifts as a set of constraints to populate an oversampled (sampled above the desired output bandwidth) processing array. The estimated subpixel locations of these samples and their values constitute a spatial domain constraint. Furthermore, the bandwidth of the alias-free image (or the sensor imposed bandwidth) is the criterion used as a spatial frequency domain constraint on the oversampled processing array. The results of testing the proposed algorithm on the simulated low- resolution forward-looking infrared (FLIR) images, real-world FLIR images, and visible images are provided. A comparison of the proposed algorithm with a standard interpolation algorithm for processing the simulated low-resolution FLIR images is also provided.

[1]  Deepu Rajan,et al.  Simultaneous Estimation of Super-Resolved Scene and Depth Map from Low Resolution Defocused Observations , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  N. K. Bose,et al.  High resolution image formation from low resolution frames using Delaunay triangulation , 2002, IEEE Trans. Image Process..

[3]  Aggelos K. Katsaggelos,et al.  Bayesian Image Estimation from an Incomplete Set of Blurred, Undersampled Low Resolution Images , 2003, IbPRIA.

[4]  Hassan Foroosh,et al.  Extension of phase correlation to subpixel registration , 2002, IEEE Trans. Image Process..

[5]  Nirmal K. Bose.  Superresolution from image sequence , 2003, 32nd Applied Imagery Pattern Recognition Workshop, 2003. Proceedings..

[6]  H Stark,et al.  High-resolution image recovery from image-plane arrays, using convex projections. , 1989, Journal of the Optical Society of America. A, Optics and image science.

[7]  R. Gerchberg.  Super-resolution through Error Energy Reduction , 1974 .

[8]  José Carlos Príncipe,et al.  Super-resolution of images based on local correlations , 1999, IEEE Trans. Neural Networks.

[9]  S. Susan Young,et al.  Superresolution performance for undersampled imagers , 2004, SPIE Defense + Commercial Sensing.

[10]  Cris L. Luengo Hendriks,et al.  Improving resolution to reduce aliasing in an undersampled image sequence , 2000 .

[11]  Sabine Süsstrunk,et al.  Superresolution images reconstructed from aliased images , 2003, Visual Communications and Image Processing.

[12]  Russell C. Hardie,et al.  Joint MAP registration and high-resolution image estimation using a sequence of undersampled images , 1997, IEEE Trans. Image Process..

[13]  Franco Gori,et al.  On an Iterative Method for Super-resolution , 1975 .

[14]  Ronald G. Driggers,et al.  Dynamic sampling, resolution enhancement, and super resolution , 2000 .

[15]  S. P. Kim,et al.  Subpixel accuracy image registration by spectrum cancellation , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[16]  Michal Irani,et al.  Image sequence enhancement using multiple motions analysis , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Zeev Zalevsky,et al.  Geometrical superresolution in infrared sensor: experimental verification , 2004 .

[18]  A. Murat Tekalp,et al.  Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time , 1997, IEEE Trans. Image Process..

[19]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[20]  Reginald L. Lagendijk,et al.  Noise reduction of image sequences using motion compensation and signal decomposition , 1995, IEEE Trans. Image Process..

[21]  A. Papoulis.  A new algorithm in spectral analysis and band-limited extrapolation. , 1975 .

[22]  Michael T. Orchard,et al.  A fast direct Fourier-based algorithm for subpixel registration of images , 2001, IEEE Trans. Geosci. Remote. Sens..

[23]  Robert L. Stevenson,et al.  Super-resolution from image sequences-a review , 1998, 1998 Midwest Symposium on Circuits and Systems (Cat. No. 98CB36268).

[24]  D. Sandwell.  BIHARMONIC SPLINE INTERPOLATION OF GEOS-3 AND SEASAT ALTIMETER DATA , 1987 .

[25]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[26]  S. Susan Young,et al.  Superresolution performance for undersampled imagers , 2005 .

[27]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[28]  D J Heeger,et al.  Model for the extraction of image flow. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[29]  Robert A. Gonsalves,et al.  Super-resolution based on low-resolution warped images , 2002, SPIE Optics + Photonics.

[30]  Shree K. Nayar,et al.  Video super-resolution using controlled subpixel detector shifts , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Mohammad S. Alam,et al.  Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames , 2000, IEEE Trans. Instrum. Meas..

[32]  Joos Vandewalle,et al.  How to take advantage of aliasing in bandlimited signals , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[33]  Daniel Gross,et al.  Improved resolution from subpixel shifted pictures , 1992, CVGIP Graph. Model. Image Process..

[34]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[35]  Terrance E. Boult,et al.  Efficient super-resolution via image warping , 2000, Image Vis. Comput..

[36]  M. Bierling,et al.  Displacement Estimation By Hierarchical Blockmatching , 1988, Other Conferences.

[37]  S. Susan Young.  Alias-free image subsampling using Fourier-based windowing methods , 2004 .

[38]  Robert L. Stevenson,et al.  Extraction of high-resolution frames from video sequences , 1996, IEEE Trans. Image Process..

[39]  Dean A. Scribner,et al.  TARID-based image super-resolution , 2002, SPIE Defense + Commercial Sensing.

[40]  Michael Elad,et al.  A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur , 2001, IEEE Trans. Image Process..

[41]  Moon Gi Kang,et al.  Regularized adaptive high-resolution image reconstruction considering inaccurate subpixel registration , 2003, IEEE Trans. Image Process..