A state-space super-resolution approach for video reconstruction

The main objective of super-resolution video reconstruction is to make use of a set of low-resolution image frames to produce their respective counterparts with higher resolution. The conventional two-equation-based Kalman filter only considers the information from the previously reconstructed high-resolution frame and the currently observed low-resolution frame for producing each high-resolution frame. It has been observed that the information inherited in the previously observed low-resolution frame could be beneficial on the reconstruction of the super-resolution video. For that, an extra observation equation is incorporated into the framework of the conventional two-equation-based Kalman filtering in this paper to establish a three-equation-based state-space approach as a more generalized framework. The closed-form solution is mathematically derived, and extensive simulations using both artificially degraded and real-life image sequences are conducted to demonstrate its superior performance. Furthermore, a unified theoretical analysis is provided to analyze the relationship between the proposed framework and two existing super-resolution approaches, the sliding-window-based Bayesian estimation approach and the conventional two-equation-based Kalman filtering, respectively.

[1]  Moon Gi Kang,et al.  Super-resolution image reconstruction , 2010, 2010 International Conference on Computer Application and System Modeling (ICCASM 2010).

[2]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[3]  Robert L. Stevenson,et al.  Simultaneous multi-frame MAP super-resolution video enhancement using spatio-temporal priors , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[4]  Aggelos K. Katsaggelos,et al.  Reconstruction of a high-resolution image by simultaneous registration, restoration, and interpolation of low-resolution images , 1995, Proceedings., International Conference on Image Processing.

[5]  Raymond H. Chan,et al.  Guest editorial , 2004, Int. J. Imaging Syst. Technol..

[6]  Gene H. Golub,et al.  Matrix computations , 1983 .

[7]  K. Riedel.  Numerical Bayesian Methods Applied to Signal Processing , 1996 .

[8]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[9]  Deepu Rajan,et al.  Generalized interpolation and its application in super-resolution imaging , 2001, Image Vis. Comput..

[10]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[11]  Andrew J. Patti,et al.  Super Resolution Video Reconstruction with Arbitrary Sampling Lattices and Non-zero Aperture Time , 1997 .

[12]  Jan Flusser,et al.  A Unified Approach to Superresolution and Multichannel Blind Deconvolution , 2007, IEEE Transactions on Image Processing.

[13]  Aggelos K. Katsaggelos,et al.  High-resolution images from low-resolution compressed video , 2003, IEEE Signal Process. Mag..

[14]  T. S. Huang,et al.  Advances in computer vision & image processing , 1988 .

[15]  Nirmal K. Bose,et al.  Recursive implementation of total least squares algorithm for image reconstruction from noisy, undersampled multiframes , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[16]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[17]  Li Meng,et al.  Subpixel Motion Estimation for Super-Resolution Image Sequence Enhancement , 1998, J. Vis. Commun. Image Represent..

[18]  A. Murat Tekalp,et al.  Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time , 1997, IEEE Trans. Image Process..

[19]  Alan C. Bovik,et al.  No-reference quality assessment using natural scene statistics: JPEG2000 , 2005, IEEE Transactions on Image Processing.

[20]  Hujun Bao,et al.  Practical super-resolution from dynamic video sequences , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[21]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[22]  Sailes K. Sengijpta.  Fundamentals of Statistical Signal Processing: Estimation Theory , 1995 .

[23]  Christian Genest,et al.  Combining Probability Distributions: A Critique and an Annotated Bibliography , 1986 .

[24]  Robert L. Stevenson,et al.  Extraction of high-resolution frames from video sequences , 1996, IEEE Trans. Image Process..

[25]  A. Murat Tekalp,et al.  Robust, object-based high-resolution image reconstruction from low-resolution video , 1997, IEEE Trans. Image Process..

[26]  Sabine Süsstrunk,et al.  A Frequency Domain Approach to Registration of Aliased Images with Application to Super-resolution , 2006, EURASIP J. Adv. Signal Process..

[27]  M. Bertero,et al.  Ill-posed problems in early vision , 1988, Proc. IEEE.

[28]  Amy R. Reibman,et al.  Quality assessment for super-resolution image enhancement , 2006, 2006 International Conference on Image Processing.

[29]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[30]  Lisimachos P. Kondi,et al.  An image super-resolution algorithm for different error levels per frame , 2006, IEEE Transactions on Image Processing.

[31]  Russell C. Hardie,et al.  Joint MAP registration and high-resolution image estimation using a sequence of undersampled images , 1997, IEEE Trans. Image Process..

[32]  Michael Elad,et al.  Fast and Robust Multi-Frame Super-Resolution , 2004, IEEE Transactions on Image Processing.

[33]  William T. Freeman,et al.  Learning low-level vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[34]  Mario Bertero,et al.  Introduction to Inverse Problems in Imaging , 1998 .

[35]  Robert L. Stevenson,et al.  Spatial Resolution Enhancement of Low-Resolution Image Sequences A Comprehensive Review with Directions for Future Research , 1998 .

[36]  S. Chaudhuri.  Super-Resolution Imaging , 2001 .

[37]  Sergios Theodoridis,et al.  A Novel Efficient Cluster-Based MLSE Equalizer for Satellite Communication Channels with-QAM Signaling , 2006, EURASIP J. Adv. Signal Process..

[38]  Michael Elad,et al.  Superresolution restoration of an image sequence: adaptive filtering approach , 1999, IEEE Trans. Image Process..

[39]  Zhaobang Pu,et al.  Comparison of different quality assessment functions in autoregulative illumination intensity algorithms , 2006 .

[40]  Aggelos K. Katsaggelos,et al.  Multi-channel Reconstruction of Video Sequences from Low-Resolution and Compressed Observations , 2003, CIARP.

[41]  Li Chen,et al.  Blind Super-Resolution Image Reconstruction using a Maximum a Posteriori Estimation , 2006, 2006 International Conference on Image Processing.

[42]  Joceli Mayer,et al.  A Robust and Computationally Efficient Simultaneous Super-Resolution Scheme for Image Sequences , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[43]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[44]  Peyman Milanfar,et al.  Super-Resolution Imaging: Analysis, Algorithms, and Applications , 2006, EURASIP J. Adv. Signal Process..