Method and architecture design for motion compensated frame interpolation in high-definition video processing

In this paper, a novel, fast, and low complexity method is proposed for motion compensated frame interpolation (MCFI). Unlike previous works involving high complexity and complicated time-consuming iterations, the proposed method adopts one-pass and low-complexity approach without any repeated iteration and is capable of dealing with high definition (HD) video processing. The proposed method employs a unique true motion engine that explores at most nine motion candidates with different motion directions and then determines one true motion by referring to neighboring spatial information as well as temporal information. Besides, the proposed method introduces an adaptive overlapped block matching algorithm to enhance the searching accuracy and visual quality. An architecture design is also proposed, which employs a modified multi-level successive eliminate algorithm (MSEA) and has capability to reduce the heavy computation of full search. Experimental results show that the proposed algorithm provides better video quality than conventional methods and shows satisfying performance for HD1080p 30fps at 180 MHz or HD720p 30fps at 83 MHz.

[1]  J.-L. Wu,et al.  Quality Enhancement of Frame Rate Up-Converted Video by Adaptive Frame Skip and Reliable Motion Extraction , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Liang-Gee Chen,et al.  Analysis and architecture design of an HDTV720p 30 frames/s H.264/AVC encoder , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  R.M. Mersereau,et al.  Motion-compensated interpolation of untransmitted frames in compressed video , 1996, Conference Record of The Thirtieth Asilomar Conference on Signals, Systems and Computers.

[4]  M. Biswas,et al.  A novel motion estimation algorithm using phase plane correlation for frame rate conversion , 2002, Conference Record of the Thirty-Sixth Asilomar Conference on Signals, Systems and Computers, 2002..

[5]  Sang Soo Kim,et al.  Motion image enhancement of LCDs , 2005, IEEE International Conference on Image Processing 2005.

[6]  Mohsen Soryani,et al.  Image segmentation and motion-adaptive frame interpolation for coding moving sequences , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[7]  Shankar Regunathan,et al.  An overview of VC-1 , 2005, Visual Communications and Image Processing.

[8]  T. Ikenaga,et al.  An efficient deblocking filter architecture with 2-dimensional parallel memory for H.264/AVC , 2005, Proceedings of the ASP-DAC 2005. Asia and South Pacific Design Automation Conference, 2005..

[9]  Truong Q. Nguyen,et al.  Optimal temporal interpolation filter for motion-compensated frame rate up conversion , 2006, IEEE Transactions on Image Processing.

[10]  Trac D. Tran,et al.  Lapped transform via time-domain pre- and post-filtering , 2003, IEEE Trans. Signal Process..

[11]  Shiqiang Yang,et al.  An adaptive frame interpolation algorithm using statistic analysis of motions and residual energy , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[12]  Michael Stegherr,et al.  Parameterizable VLSI architectures for the full-search block-matching algorithm , 1989 .

[13]  Xiqi Gao,et al.  A multilevel successive elimination algorithm for block matching motion estimation , 2000, IEEE Trans. Image Process..

[14]  Chen-Yi Lee,et al.  An In/Post-Loop Deblocking Filter With Hybrid Filtering Schedule , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Tao Chen.  Adaptive temporal interpolation using bidirectional motion estimation and compensation , 2002, Proceedings. International Conference on Image Processing.

[16]  P. Lakamsani.  An architecture for enhanced three step search generalized for hierarchical motion estimation algorithms , 1997 .

[17]  Jani Lainema,et al.  Adaptive deblocking filter , 2003, IEEE Trans. Circuits Syst. Video Technol..

[18]  Truong Q. Nguyen,et al.  Correlation-Based Motion Vector Processing With Adaptive Interpolation Scheme for Motion-Compensated Frame Interpolation , 2009, IEEE Transactions on Image Processing.

[19]  Nikolaos Grammalidis,et al.  Temporal frame interpolation for stereoscopic sequences using object-based motion estimation and occlusion detection , 1995, Proceedings., International Conference on Image Processing.

[20]  Youn-Long Lin,et al.  A near optimal deblocking filter for H.264 advanced video coding , 2006, Asia and South Pacific Conference on Design Automation, 2006..

[21]  Ezzatollah Salari,et al.  Successive elimination algorithm for motion estimation , 1995, IEEE Trans. Image Process..

[22]  Liang-Gee Chen,et al.  Architecture design for deblocking filter in H.264/JVT/AVC , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[23]  John W. Woods,et al.  Frame-rate up-conversion using transmitted motion and segmentation fields for very low bit-rate video coding , 1997, Proceedings of International Conference on Image Processing.

[24]  Roberto Castagno,et al.  A method for motion adaptive frame rate up-conversion , 1996, IEEE Trans. Circuits Syst. Video Technol..

[25]  Gerard de Haan,et al.  True-motion estimation with 3-D recursive search block matching , 1993, IEEE Trans. Circuits Syst. Video Technol..

[26]  Chang-Su Kim,et al.  Motion-Compensated Frame Interpolation Using Bilateral Motion Estimation and Adaptive Overlapped Block Motion Compensation , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Jiang Li,et al.  A low complexity motion compensated frame interpolation method , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[28]  Aidong Men,et al.  A method for halo artifact reduction in MEMC , 2009, 2009 Digest of Technical Papers International Conference on Consumer Electronics.

[29]  Hitoshi Kiya,et al.  A fast full search block matching algorithm for MPEG-4 video , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[30]  Rae-Hong Park,et al.  Coarse-to-fine frame interpolation for frame rate up-conversion using pyramid structure , 2003, IEEE Trans. Consumer Electron..

[31]  Wei Zhang,et al.  An efficient architecture for adaptive deblocking filter of H.264/AVC video coding , 2004, IEEE Trans. Consumer Electron..

[32]  Sun-Yuan Kung,et al.  Frame-rate up-conversion using transmitted true motion vectors , 1998, 1998 IEEE Second Workshop on Multimedia Signal Processing (Cat. No.98EX175).

[33]  Taehyeun Ha,et al.  Motion compensated frame interpolation by new block-based motion estimation algorithm , 2004, IEEE Transactions on Consumer Electronics.

[34]  Sung-Jea Ko,et al.  Frame rate up-conversion using perspective transform , 2006, IEEE Transactions on Consumer Electronics.

[35]  Tian-Sheuan Chang,et al.  An in-place architecture for the deblocking filter in H.264/AVC , 2006, IEEE Trans. Circuits Syst. II Express Briefs.

[36]  JongWon Kim,et al.  Motion-compensated frame interpolation scheme for H.263 codec , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[37]  A. Taguchi,et al.  Motion-compensated frame rate up-conversion based on block matching algorithm with multi-size blocks , 2005, 2005 International Symposium on Intelligent Signal Processing and Communication Systems.

[38]  V. Munoz-Jimenez,et al.  Bidirectional Motion Estimation Approach Using Warping Mesh Combined to Frame Interpolation , 2008, 2008 IEEE International Symposium on Signal Processing and Information Technology.

[39]  Truong Q. Nguyen,et al.  LCD Motion Blur Reduction: A Signal Processing Approach , 2008, IEEE Transactions on Image Processing.

[40]  Wen-Hsiao Peng,et al.  A platform based bus-interleaved architecture for de-blocking filter in H.264/MPEG-4 AVC , 2005, IEEE Transactions on Consumer Electronics.

[41]  Chen-Yi Lee,et al.  A memory-efficient deblocking filter for H.264/AVC video coding , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[42]  Zongliang Gan,et al.  Motion compensated frame interpolation based on H.264 decoder , 2007 .

[43]  Truong Q. Nguyen,et al.  A Multistage Motion Vector Processing Method for Motion-Compensated Frame Interpolation , 2008, IEEE Transactions on Image Processing.