Pyramid‐based super‐resolution of the undersampled and subpixel shifted image sequence

The existing methods for the reconstruction of a super‐resolution image from the undersampled and subpixel shifted image sequence have to solve a large ill‐condition equation group by approximately finding the inverse matrix or performing many iterations to approach the solution. The former leads to a big burden of computation, and the latter causes the artifacts or noise to be overstressed. So they are rarely implemented in practical use. In order to solve these problems, in this article, we consider applying pyramid structure to the super‐resolution of the image sequence and present a suitable pyramid framework, called Super‐Resolution Image Pyramid (SRIP), and determine the pyramid back‐projection. Pyramid structure and methods are widely used in image processing and computer vision. But we have not found their applications to the super‐resolution in literatures. We give a complete description for SRIP. As an example, the Iterative Back‐Projection (IBP) suggested by Peleg (1991, 1993) is integrated in this pyramid framework. The experiments and the error analysis are performed to show the effectiveness of this framework. The image resolution can be improved better even in the case of severely undersampled images. In addition, the other general super‐resolution methods can be easily integrated in this framework so that they can be done in parallel so as to meet the need of real‐time processing. © 2003 Wiley Periodicals, Inc. Int J Imaging Syst Technol 12, 254–263, 2002; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ima.10033

[1]  Aggelos K. Katsaggelos,et al.  Reconstruction of a high-resolution image from multiple-degraded misregistered low-resolution images , 1994, Other Conferences.

[2]  H Stark,et al.  High-resolution image recovery from image-plane arrays, using convex projections. , 1989, Journal of the Optical Society of America. A, Optics and image science.

[3]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[4]  Josef Bigün,et al.  Hierarchical image segmentation by multi-dimensional clustering and orientation-adaptive boundary refinement , 1995, Pattern Recognit..

[5]  Michael Unser,et al.  An improved least squares Laplacian pyramid for image compression , 1992, Signal Process..

[6]  Charles A. Bouman,et al.  Multiple Resolution Segmentation of Textured Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Michal Irani,et al.  Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency , 1993, J. Vis. Commun. Image Represent..

[8]  Nirmal K. Bose,et al.  Recursive reconstruction of high resolution image from noisy undersampled multiframes , 1990, IEEE Trans. Acoust. Speech Signal Process..

[9]  C. Morandi,et al.  Registration of Translated and Rotated Images Using Finite Fourier Transforms , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  P. Schroeter,et al.  Hierarchical Robust Motion Estimation for Segmentation of Moving Objects , 1993 .

[11]  Eiiti Takizawa,et al.  On the Generalized Sampling Theorem (非線型振動理論の研究会報告集) , 1968 .

[12]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[13]  Jr. J. L. Brown.  Multi-channel sampling of low-pass signals , 1981 .

[14]  Russell C. Hardie,et al.  Joint MAP registration and high-resolution image estimation using a sequence of undersampled images , 1997, IEEE Trans. Image Process..

[15]  Martin Vetterli,et al.  Interpolative multiresolution coding of advance television with compatible subchannels , 1991, IEEE Trans. Circuits Syst. Video Technol..

[16]  A. Papoulis,et al.  Generalized sampling expansion , 1977 .

[17]  Michael Elad,et al.  Super-Resolution Reconstruction of Image Sequences , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Daniel Gross,et al.  Improved resolution from subpixel shifted pictures , 1992, CVGIP Graph. Model. Image Process..

[19]  Michael Unser,et al.  A pyramid approach to subpixel registration based on intensity , 1998, IEEE Trans. Image Process..

[20]  Michael Unser,et al.  On the optimality of ideal filters for pyramid and wavelet signal approximation , 1993, IEEE Trans. Signal Process..

[21]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[22]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[23]  R A Schowengerdt,et al.  Image sampling, reconstruction, and the effect of sample-scene phasing. , 1982, Applied optics.