Super-resolution of video using key frames and motion estimation

Many scalable video coding systems use frame down- sampling in order to reduce complexity and to enable enhancement layers. Super-resolution (SR) can be used to help the up-sampling and recovering processes of those frames. We are interested in reversed-complexity (distributed) coding methods, wherein few key frames are encoded at normal resolution, while the rest are down- sampled and encoded at reduced resolution along with the enhancement layers. We are only interested on the decoder side, wherein we carry motion estimation of the down- sampled frames using the key frames as references. When a match is made, the high-frequency components of the key frames (KF) are used to super-resolve the non-key frames (NKF). The motion estimation process is performed using blocks of band-pass versions of the frames, rather than low- pass ones. Results indicate the improved performance of the proposed super-resolution algorithm.

[1]  Jerome M. Shapiro,et al.  Embedded image coding using zerotrees of wavelet coefficients , 1993, IEEE Trans. Signal Process..

[2]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[3]  Aggelos K. Katsaggelos,et al.  High-resolution images from low-resolution compressed video , 2003, IEEE Signal Process. Mag..

[4]  Yaron Caspi,et al.  Under the supervision of , 2003 .

[5]  Alberto Signoroni,et al.  A fully scalable video coder with inter-scale wavelet prediction and morphological coding , 2005, Visual Communications and Image Processing.

[6]  Bernd Girod,et al.  Distributed Video Coding , 2005, Proceedings of the IEEE.

[7]  Edward J. Delp,et al.  Wyner-Ziv video side estimator: conventional motion search methods revisited , 2005, IEEE International Conference on Image Processing 2005.

[8]  Bruno Macchiavello,et al.  A simple reversed-complexity Wyner-Ziv video coding mode based on a spatial reduction framework , 2007, Electronic Imaging.

[9]  Heiko Schwarz,et al.  Overview of the Scalable Video Coding Extension of the H.264/AVC Standard , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Aggelos K. Katsaggelos,et al.  Super Resolution of Images and Video , 2006, Super Resolution of Images and Video.

[11]  Bruno Macchiavello,et al.  Motion-Based Side-Information Generation for a Scalable Wyner-Ziv Video Coder , 2007, 2007 IEEE International Conference on Image Processing.

[12]  Debargha Mukherjee,et al.  Super resolution of video using key frames , 2008, 2008 IEEE International Symposium on Circuits and Systems.