Quality Enhancement of Frame Rate Up-Converted Video by Adaptive Frame Skip and Reliable Motion Extraction

Frame rate up-conversion is a postprocessing tool to convert the frame rate from a lower number to a higher one. It is a useful technique for a lot of practical applications, such as display format conversion, low bit rate video coding and slow motion playback. Unlike traditional approaches, such as frame repetition or linear frame interpolation, motion-compensated frame interpolation (MCFI) technique is more efficient since it takes block motion into account. In this paper, by considering the deficiencies of previous works, new criteria and coding schemes for improving motion derivation and interpolation processes are proposed. Next, for video coding applications, adaptive frame skip is executed at the encoder side to maximize the power of MCFI so that the quality of interpolated frames is guaranteed. Experimental results show that our proposal effectively enhances the overall quality of the frame rate up-converted video sequence, both subjectively and objectively.

[1]  A. Murat Tekalp,et al.  Digital Video Processing , 1995 .

[2]  Mohammad Umar Siddiqi,et al.  Low rate video frame interpolation - challenges and solution , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[3]  Rae-Hong Park,et al.  Weighted-adaptive motion-compensated frame rate up-conversion , 2003, IEEE Trans. Consumer Electron..

[4]  Bede Liu,et al.  Recovery of lost or erroneously received motion vectors , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[5]  Jiang Li,et al.  A low complexity motion compensated frame interpolation method , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[6]  JongWon Kim,et al.  Global/local motion-compensated frame interpolation for low-bit-rate video , 2000, Electronic Imaging.

[7]  Tie-Yan Liu,et al.  Frame interpolation scheme using inertia motion prediction , 2003, Signal Process. Image Commun..

[8]  K. Kawaguchi,et al.  Frame rate up-conversion considering multiple motion , 1997, Proceedings of International Conference on Image Processing.

[9]  C.-C. Jay Kuo,et al.  Motion-compensated interpolation for low-bit-rate video quality enhancement , 1998, Optics & Photonics.

[10]  Oscar C. Au,et al.  Temporal interpolation using wavelet domain motion estimation and motion compensation , 2002, Proceedings. International Conference on Image Processing.

[11]  Victor H. S. Ha,et al.  Portable receivers for digital multimedia broadcasting , 2004, IEEE Transactions on Consumer Electronics.

[12]  Rae-Hong Park,et al.  Coarse-to-fine frame interpolation for frame rate up-conversion using pyramid structure , 2003, IEEE Trans. Consumer Electron..

[13]  Tao Chen.  Adaptive temporal interpolation using bidirectional motion estimation and compensation , 2002, Proceedings. International Conference on Image Processing.

[14]  Jaeseok Kim,et al.  Motion compensated frame interpolation by new block-based motion estimation algorithm , 2004, IEEE Trans. Consumer Electron..

[15]  Sung-Jea Ko,et al.  New frame rate up-conversion using bi-directional motion estimation , 2000, IEEE Trans. Consumer Electron..

[16]  JongWon Kim,et al.  Hybrid global-local motion compensated frame interpolation for low bit rate video coding , 2003, J. Vis. Commun. Image Represent..

[17]  O. C. Au,et al.  Comparison between block-based and pixel-based temporal interpolation for video coding , 1998, ISCAS '98. Proceedings of the 1998 IEEE International Symposium on Circuits and Systems (Cat. No.98CH36187).

[18]  Sun-Yuan Kung,et al.  Frame-rate up-conversion using transmitted true motion vectors , 1998, 1998 IEEE Second Workshop on Multimedia Signal Processing (Cat. No.98EX175).

[19]  Satoshi Kondo,et al.  Frame-rate up-conversion using reliable analysis of transmitted motion information , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[20]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[21]  Pierre Moulin,et al.  Frame interpolation and bidirectional prediction of video using compactly encoded optical-flow fields and label fields , 1999, IEEE Trans. Circuits Syst. Video Technol..

[22]  Yongmin Kim,et al.  Using motion-compensated frame-rate conversion for the correction of 3: 2 pulldown artifacts in video sequences , 2000, IEEE Trans. Circuits Syst. Video Technol..

[23]  Roberto Castagno,et al.  A method for motion adaptive frame rate up-conversion , 1996, IEEE Trans. Circuits Syst. Video Technol..

[24]  Mohammed E. Al-Mualla.  Motion field interpolation for frame rate conversion , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[25]  JongWon Kim,et al.  MCI-embedded motion-compensated prediction for quality enhancement of frame interpolation , 2001, SPIE Optics East.

[26]  JongWon Kim,et al.  Nonlinear motion-compensated interpolation for low-bit-rate video , 2000, SPIE Optics + Photonics.

[27]  JongWon Kim,et al.  Motion-compensated frame interpolation scheme for H.263 codec , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[28]  Reginald L. Lagendijk,et al.  Motion compensated frame rate conversion of motion pictures , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.