Video superresolution reconstruction based on subpixel registration and iterative back projection

To improve the spatial resolution of video, a superresolution reconstruction method based on a sliding window is proposed utilizing the movement information between frames in the low-resolution video. We propose a registration algorithm based on a four-parameter transformation model through Taylor series expansion, using an iterative solving method as well as the Gaussian pyramid image model to estimate the movement parameters from coarseness to fine. Superresolution frames are reconstructed using an iterative back projection (IBP) algorithm. We also present the suitable length of the sliding window and the reasonable iteration number of the IBP algorithm in the video superresolution reconstruction. Our algorithm is compared to other algorithms on simulated images and actual color videos. Both show that our registration algorithm achieves higher subpixel accuracy than other algorithms, even in the case of large movements, and that the reconstructed video has better visual effects and stronger resolution ability. It can be extensively applied to the superresolution reconstruction of video sequences in which the frames are different from each other mainly by translation and rotation.

[1]  Hassan Foroosh,et al.  Extension of phase correlation to subpixel registration , 2002, IEEE Trans. Image Process..

[2]  Shree K. Nayar,et al.  Video super-resolution using controlled subpixel detector shifts , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Michal Irani,et al.  Computing occluding and transparent motions , 1994, International Journal of Computer Vision.

[4]  Shmuel Peleg,et al.  Robust super-resolution , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[5]  Michael Elad,et al.  Advances and challenges in super‐resolution , 2004, Int. J. Imaging Syst. Technol..

[6]  Sabine Süsstrunk,et al.  A Frequency Domain Approach to Registration of Aliased Images with Application to Super-resolution , 2006, EURASIP J. Adv. Signal Process..

[7]  Sukhendu Das,et al.  Indoor versus Outdoor Scene Classification Using Probabilistic Neural Network , 2007, EURASIP J. Adv. Signal Process..

[8]  Luca Lucchese,et al.  A noise-robust frequency domain technique for estimating planar roto-translations , 2000, IEEE Trans. Signal Process..

[9]  Andrea Fusiello,et al.  High resolution video mosaicing with global alignment , 2004, CVPR 2004.

[10]  Michael Elad,et al.  Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images , 1997, IEEE Trans. Image Process..

[11]  Kenneth E. Barner,et al.  A Computationally Efficient Super-Resolution Algorithm for Video Processing Using Partition Filters , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Thomas S. Huang,et al.  Ordinal Regression Based Subpixel Shift Estimation for Video Super-Resolution , 2007, EURASIP J. Adv. Signal Process..

[13]  Weisi Lin,et al.  Improved Super-Resolution Reconstruction From Video , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Yitzhak Yitzhaky,et al.  Superresolution reconstruction of a video captured by a vibrated time delay and integration camera , 2006, J. Electronic Imaging.

[15]  Shmuel Peleg,et al.  Image sequence enhancement using sub-pixel displacements , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[17]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[18]  Joshua Gluckman,et al.  Gradient field distributions for the registration of images , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[19]  Michael T. Orchard,et al.  A fast direct Fourier-based algorithm for subpixel registration of images , 2001, IEEE Trans. Geosci. Remote. Sens..

[20]  Takeo Kanade,et al.  Image Registration Using Wavelet-Based Motion Model , 2000, International Journal of Computer Vision.

[21]  Yaron Caspi,et al.  Under the supervision of , 2003 .

[22]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[23]  Michael Elad,et al.  Video-to-Video Dynamic Super-Resolution for Grayscale and Color Sequences , 2006, EURASIP J. Adv. Signal Process..

[24]  Takeo Kanade,et al.  Limits on Super-Resolution and How to Break Them , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[26]  A. Murat Tekalp,et al.  Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time , 1997, IEEE Trans. Image Process..

[27]  Klamer Schutte,et al.  Robust Fusion of Irregularly Sampled Data Using Adaptive Normalized Convolution , 2006, EURASIP J. Adv. Signal Process..

[28]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..