High-resolution images from low-resolution compressed video

Surveys the field of super resolution (SR) processing for compressed video. The introduction of motion vectors, compression noise, and additional redundancies within the image sequence makes this problem fertile ground for novel processing methods. In conducting this survey, though, we develop and present all techniques within the Bayesian framework. This adds consistency to the presentation and facilitates comparison between the different methods. The article is organized as follows. We define the acquisition system utilized by the surveyed procedures. Then we formulate the HR problem within the Bayesian framework and survey models for the acquisition and compression systems. This requires consideration of both the motion vectors and transform coefficients within the compressed bit stream. We survey models for the original HR image intensities and displacement values. We discuss solutions for the SR problem and provide examples of several approaches.

[1]  Jesús Chamorro,et al.  Desarrollo de modelos computacionales de representación de secuencias de imágenes y su aplicación a la estimación del movimiento , 2001 .

[2]  Robert L. Stevenson,et al.  DCT quantization noise in compressed images , 2001, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  W. Clem Karl,et al.  Efficient multiscale regularization with applications to the computation of optical flow , 1994, IEEE Trans. Image Process..

[4]  Aggelos K. Katsaggelos,et al.  Bayesian resolution enhancement of compressed video , 2004, IEEE Transactions on Image Processing.

[5]  Aggelos K. Katsaggelos,et al.  Reconstruction of high-resolution image frames from a sequence of low-resolution and compressed observations , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[6]  Andrew Zisserman,et al.  Super-resolution from multiple views using learnt image models , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[7]  Robert L. Stevenson,et al.  Spatial Resolution Enhancement of Low-Resolution Image Sequences A Comprehensive Review with Directions for Future Research , 1998 .

[8]  Yücel Altunbasak,et al.  Super-resolution image estimation for transform coded video with application to MPEG , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[9]  N. Galatsanos,et al.  Multichannel regularized recovery of compressed video sequences , 2001 .

[10]  Richard R. Schultz,et al.  Extraction of high-resolution video stills from MPEG image sequences , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[11]  Yücel Altunbasak,et al.  A maximum a posteriori estimator for high resolution video reconstruction from MPEG video , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[12]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[13]  Oscar Nestares,et al.  Probabilistic estimation of optical flow in multiple band-pass directional channels , 2001, Image Vis. Comput..

[14]  Arun N. Netravali,et al.  Digital Pictures: Representation and Compression , 1988 .

[15]  Michal Irani,et al.  Computing occluding and transparent motions , 1994, International Journal of Computer Vision.

[16]  Yücel Altunbasak,et al.  Super-resolution still and video reconstruction from MPEG-coded video , 2002, IEEE Trans. Circuits Syst. Video Technol..

[17]  Michal Irani,et al.  Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency , 1993, J. Vis. Commun. Image Represent..

[18]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[19]  Aggelos K. Katsaggelos,et al.  Super-Resolution from Compressed Video , 2002 .

[20]  Edward H. Adelson,et al.  Probability distributions of optical flow , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Edward H. Adelson,et al.  A unified mixture framework for motion segmentation: incorporating spatial coherence and estimating the number of models , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  D. Youla,et al.  Image Restoration by the Method of Convex Projections: Part 1ߞTheory , 1982, IEEE Transactions on Medical Imaging.

[23]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[24]  Aggelos K. Katsaggelos,et al.  Resolution enhancement of compressed low resolution video , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[25]  Konstantinos Konstantinides,et al.  Image and video compression standards , 1995 .

[26]  Moon Gi Kang,et al.  Spatially adaptive high-resolution image reconstruction of low-resolution DCT-based compressed images , 2002, Proceedings. International Conference on Image Processing.

[27]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[28]  Aggelos K. Katsaggelos,et al.  Bayesian high-resolution reconstruction of low-resolution compressed video , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[29]  Aggelos K. Katsaggelos,et al.  Spatially adaptive high-resolution image reconstruction of DCT-based compressed images , 2004, IEEE Transactions on Image Processing.

[30]  Eric Dubois,et al.  Bayesian Estimation of Motion Vector Fields , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  B. Gunturk,et al.  Multiframe resolution-enhancement methods for compressed video , 2002, IEEE Signal Processing Letters.

[32]  Yücel Altunbasak,et al.  Bayesian resolution-enhancement framework for transform-coded video , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[33]  Aggelos K. Katsaggelos,et al.  Simultaneous motion estimation and resolution enhancement of compressed low resolution video , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[34]  Aggelos K. Katsaggelos,et al.  High-resolution image reconstruction of low-resolution DCT-based compressed images , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.